[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:00:22 dist-test-slave-x4qh krb5kdc[5561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:00:22 dist-test-slave-x4qh krb5kdc[5561](info): set up 2 sockets
Aug 06 01:00:22 dist-test-slave-x4qh krb5kdc[5561](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 01:00:24.200723 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.005s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:00:24 dist-test-slave-x4qh krb5kdc[5561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:00:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:00:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:00:24.373261 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:42087
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:42087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:24.716678 5576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:24.717016 5576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:24.717227 5576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:24.724150 5576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:00:24.724267 5576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:24.724375 5576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:24.724474 5576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:00:24.724558 5576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:00:24.734040 5576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:42087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:42087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:24.735741 5576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:24.737382 5576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:24.754329 5584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:24.754602 5583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:24.754987 5586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:24.755101 5576 server_base.cc:1034] running on GCE node
I20260806 01:00:24.756171 5576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:24.758076 5576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:24.759281 5576 hybrid_clock.cc:648] HybridClock initialized: now 1785978024759229 us; error 73 us; skew 500 ppm
I20260806 01:00:24.759773 5576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:24 dist-test-slave-x4qh krb5kdc[5561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978024, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:24.763039 5576 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:00:24.765286 5576 webserver.cc:460] Webserver started at http://127.5.108.190:43629/ using document root <none> and password file <none>
I20260806 01:00:24.766306 5576 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:24.766448 5576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:24.766884 5576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:24.770505 5576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "6cc0b2dba05e4721800dd13d907eab5b"
format_stamp: "Formatted at 2026-08-06 01:00:24 on dist-test-slave-x4qh"
server_key: "01e08b3027a61322dd6230370dfb6e05"
server_key_iv: "a0f8f5db2e2e744523cfbb4982bd65a0"
server_key_version: "encryptionkey@0"
I20260806 01:00:24.771256 5576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "6cc0b2dba05e4721800dd13d907eab5b"
format_stamp: "Formatted at 2026-08-06 01:00:24 on dist-test-slave-x4qh"
server_key: "01e08b3027a61322dd6230370dfb6e05"
server_key_iv: "a0f8f5db2e2e744523cfbb4982bd65a0"
server_key_version: "encryptionkey@0"
I20260806 01:00:24.778230 5576 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:00:24.783469 5594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:24.785290 5576 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:00:24.785560 5576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "6cc0b2dba05e4721800dd13d907eab5b"
format_stamp: "Formatted at 2026-08-06 01:00:24 on dist-test-slave-x4qh"
server_key: "01e08b3027a61322dd6230370dfb6e05"
server_key_iv: "a0f8f5db2e2e744523cfbb4982bd65a0"
server_key_version: "encryptionkey@0"
I20260806 01:00:24.785774 5576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:24.802619 5576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:24.807302 5576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:24.807695 5576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:24.830132 5576 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:42087
I20260806 01:00:24.830165 5677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:42087 every 8 connection(s)
I20260806 01:00:24.832322 5576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:00:24.838281 5678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:24.841321 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5576
I20260806 01:00:24.841779 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:00:24.844627 5554 external_mini_cluster.cc:1428] Setting key 2bcaa11a0d8c3908f7481a1d27d1442f
I20260806 01:00:24.856750 5678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b: Bootstrap starting.
I20260806 01:00:24.861466 5678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:24.863487 5678 log.cc:824] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:24.867889 5678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b: No bootstrap required, opened a new log
I20260806 01:00:24.878129 5678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc0b2dba05e4721800dd13d907eab5b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42087 } }
I20260806 01:00:24.878651 5678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:24.878855 5678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc0b2dba05e4721800dd13d907eab5b, State: Initialized, Role: FOLLOWER
Aug 06 01:00:24 dist-test-slave-x4qh krb5kdc[5561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:24.879952 5678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc0b2dba05e4721800dd13d907eab5b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42087 } }
I20260806 01:00:24.880247 5678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:00:24.880539 5678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:00:24.880831 5678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:24.884402 5678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc0b2dba05e4721800dd13d907eab5b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42087 } }
I20260806 01:00:24.885480 5678 leader_election.cc:302] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cc0b2dba05e4721800dd13d907eab5b; no voters:
I20260806 01:00:24.886363 5678 leader_election.cc:288] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:00:24.886556 5689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:24.889707 5678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:00:24.891806 5689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [term 1 LEADER]: Becoming Leader. State: Replica: 6cc0b2dba05e4721800dd13d907eab5b, State: Running, Role: LEADER
I20260806 01:00:24.892616 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:24.851748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.5.108.190:42087)
0806 01:00:24.852961 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:24.852978 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:24.854798 (+ 1820us) server_negotiation.cc:407] Connection header received
0806 01:00:24.864917 (+ 10119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:24.864935 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:24.865551 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:24.866004 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:24.868903 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:24.871389 (+ 2486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:24.873216 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:24.874330 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:24.874567 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:24.880679 (+ 6112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:24.880747 (+ 68us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:24.880790 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:24.880840 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:24.884977 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:24.885861 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:24.885867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:24.885872 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:24.886093 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:24.886601 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:24.886604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:24.886606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:24.887550 (+ 944us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:24.890534 (+ 2984us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:24.890750 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 01:00:24.891017 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":443,"threads_started":1}
I20260806 01:00:24.892720 5689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc0b2dba05e4721800dd13d907eab5b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42087 } }
I20260806 01:00:24.900694 5692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cc0b2dba05e4721800dd13d907eab5b. Latest consensus state: current_term: 1 leader_uuid: "6cc0b2dba05e4721800dd13d907eab5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc0b2dba05e4721800dd13d907eab5b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42087 } } }
I20260806 01:00:24.901013 5692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:24.901587 5691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cc0b2dba05e4721800dd13d907eab5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc0b2dba05e4721800dd13d907eab5b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42087 } } }
I20260806 01:00:24.901847 5691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:24.903764 5703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:00:24.911693 5703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:00:24.926263 5703 catalog_manager.cc:1347] Generated new cluster ID: 72a5d1498ff5429fb45b06b0cf4d3136
I20260806 01:00:24.926388 5703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:00:24.955807 5703 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:00:24.957485 5703 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:00:24.963027 5703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cc0b2dba05e4721800dd13d907eab5b: Generated new TSK 0
I20260806 01:00:24.963979 5703 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:00:25.014176 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42087
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:25.342839 5719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:25.343124 5719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:25.343308 5719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:25.350337 5719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:25.350452 5719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:25.350625 5719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:00:25.360170 5719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:25.361383 5719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:25.362723 5719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:25.376701 5730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:25.376742 5726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:25.376793 5725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:25.377550 5719 server_base.cc:1034] running on GCE node
I20260806 01:00:25.378158 5719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:25.379007 5719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:25.380194 5719 hybrid_clock.cc:648] HybridClock initialized: now 1785978025380140 us; error 49 us; skew 500 ppm
I20260806 01:00:25.380565 5719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:25 dist-test-slave-x4qh krb5kdc[5561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978025, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:25.383699 5719 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:00:25.385084 5719 webserver.cc:460] Webserver started at http://127.5.108.129:43517/ using document root <none> and password file <none>
I20260806 01:00:25.385789 5719 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:25.385941 5719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:25.386262 5719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:25.389299 5719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "83bb0c23cf3141bc850c1ca510d76142"
format_stamp: "Formatted at 2026-08-06 01:00:25 on dist-test-slave-x4qh"
server_key: "21e8f24786a98a4839121c5a52a0d5e4"
server_key_iv: "cee38048504b76a19d26d927541e08c7"
server_key_version: "encryptionkey@0"
I20260806 01:00:25.390269 5719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "83bb0c23cf3141bc850c1ca510d76142"
format_stamp: "Formatted at 2026-08-06 01:00:25 on dist-test-slave-x4qh"
server_key: "21e8f24786a98a4839121c5a52a0d5e4"
server_key_iv: "cee38048504b76a19d26d927541e08c7"
server_key_version: "encryptionkey@0"
I20260806 01:00:25.395529 5719 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:00:25.398854 5740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:25.400084 5719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:00:25.400254 5719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "83bb0c23cf3141bc850c1ca510d76142"
format_stamp: "Formatted at 2026-08-06 01:00:25 on dist-test-slave-x4qh"
server_key: "21e8f24786a98a4839121c5a52a0d5e4"
server_key_iv: "cee38048504b76a19d26d927541e08c7"
server_key_version: "encryptionkey@0"
I20260806 01:00:25.400476 5719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:25.417112 5719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:25.420817 5719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:25.421095 5719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:25.422115 5719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:25.423723 5719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:25.423805 5719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:25.424016 5719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:25.424077 5719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:25.458086 5719 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:46725
I20260806 01:00:25.458158 5901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:46725 every 8 connection(s)
I20260806 01:00:25.459313 5719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:00:25.469652 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5719
I20260806 01:00:25.469884 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:00:25.470654 5554 external_mini_cluster.cc:1428] Setting key 0bc2d86dac83a06213383670788affce
Aug 06 01:00:25 dist-test-slave-x4qh krb5kdc[5561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978025, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:25.480455 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:25.462492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:49125 (local address 127.5.108.190:42087)
0806 01:00:25.462955 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:25.462960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:25.463927 (+ 967us) server_negotiation.cc:407] Connection header received
0806 01:00:25.465870 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:25.465875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:25.465962 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:25.466083 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:25.467607 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:25.468333 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:25.470320 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:25.470741 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:25.470901 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:25.474789 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:25.474793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:25.474796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:25.474843 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:25.477440 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:25.478135 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:25.478142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:25.478150 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:25.478267 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:25.478687 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:25.478692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:25.478696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:25.478952 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:25.479110 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:25.479825 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 01:00:25.480084 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":270,"threads_started":1}
I20260806 01:00:25.481312 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:25.462965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:42087 (local address 127.5.108.129:49125)
0806 01:00:25.463772 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:25.463815 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:25.465557 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:25.466322 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:25.466329 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:25.466844 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:25.467431 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:25.467464 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:25.468513 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:25.468517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:25.470172 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:25.470188 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:25.470989 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:25.470999 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:25.471229 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:25.472023 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:25.472055 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:25.474595 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:25.477581 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:25.477589 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:25.477602 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:25.478007 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:25.478378 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:25.478383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:25.478386 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:25.478567 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:25.479082 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:25.479089 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:25.479458 (+ 369us) client_negotiation.cc:769] Sending connection context
0806 01:00:25.479709 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 01:00:25.480219 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":362,"threads_started":1}
I20260806 01:00:25.482901 5902 heartbeater.cc:347] Connected to a master server at 127.5.108.190:42087
I20260806 01:00:25.483388 5902 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:25.484510 5902 heartbeater.cc:511] Master 127.5.108.190:42087 requested a full tablet report, sending...
I20260806 01:00:25.489038 5621 ts_manager.cc:194] Registered new tserver with Master: 83bb0c23cf3141bc850c1ca510d76142 (127.5.108.129:46725)
I20260806 01:00:25.492125 5621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:49125
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:00:25.518604 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42087
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:25.875674 5908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:25.875942 5908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:25.876122 5908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:25.883188 5908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:25.883311 5908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:25.883509 5908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:00:25.893332 5908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:25.894613 5908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:25.896023 5908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:25.909770 5915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:25.909765 5918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:25.909801 5913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:25.910782 5908 server_base.cc:1034] running on GCE node
I20260806 01:00:25.911516 5908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:25.912484 5908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:25.913722 5908 hybrid_clock.cc:648] HybridClock initialized: now 1785978025913656 us; error 38 us; skew 500 ppm
I20260806 01:00:25.914085 5908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:25 dist-test-slave-x4qh krb5kdc[5561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978025, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:25.917325 5908 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:00:25.918941 5908 webserver.cc:460] Webserver started at http://127.5.108.130:33817/ using document root <none> and password file <none>
I20260806 01:00:25.919706 5908 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:25.919865 5908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:25.920217 5908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:25.923240 5908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "b745980ca96c42ceaa0a86966a99d197"
format_stamp: "Formatted at 2026-08-06 01:00:25 on dist-test-slave-x4qh"
server_key: "52f44d50d5598ba14541a1391f521aff"
server_key_iv: "02ed97dc1ed5114f2ff75bf69d1d85c3"
server_key_version: "encryptionkey@0"
I20260806 01:00:25.924178 5908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "b745980ca96c42ceaa0a86966a99d197"
format_stamp: "Formatted at 2026-08-06 01:00:25 on dist-test-slave-x4qh"
server_key: "52f44d50d5598ba14541a1391f521aff"
server_key_iv: "02ed97dc1ed5114f2ff75bf69d1d85c3"
server_key_version: "encryptionkey@0"
I20260806 01:00:25.929504 5908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:00:25.933017 5927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:25.934273 5908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:00:25.934465 5908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "b745980ca96c42ceaa0a86966a99d197"
format_stamp: "Formatted at 2026-08-06 01:00:25 on dist-test-slave-x4qh"
server_key: "52f44d50d5598ba14541a1391f521aff"
server_key_iv: "02ed97dc1ed5114f2ff75bf69d1d85c3"
server_key_version: "encryptionkey@0"
I20260806 01:00:25.934702 5908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:25.968338 5908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:25.972426 5908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:25.972748 5908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:25.973712 5908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:25.975147 5908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:25.975267 5908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:25.975445 5908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:25.975522 5908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:26.016552 5908 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:40479
I20260806 01:00:26.016593 6096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:40479 every 8 connection(s)
I20260806 01:00:26.017926 5908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:00:26.026029 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5908
I20260806 01:00:26.026248 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:00:26.026691 5554 external_mini_cluster.cc:1428] Setting key 78de677aff73a18b6f6b8b13357830d5
Aug 06 01:00:26 dist-test-slave-x4qh krb5kdc[5561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978025, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:26.038478 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:26.020507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:44645 (local address 127.5.108.190:42087)
0806 01:00:26.020974 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:26.020978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:26.021845 (+ 867us) server_negotiation.cc:407] Connection header received
0806 01:00:26.023708 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:26.023714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:26.023802 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:26.023911 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:26.025420 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.026160 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.027962 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.028460 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.028570 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:26.032538 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:26.032542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:26.032545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:26.032572 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:26.035251 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.036138 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.036141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.036143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.036259 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.036762 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.036765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.036767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.037000 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:26.037105 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:26.037961 (+ 856us) server_negotiation.cc:299] Negotiation successful
0806 01:00:26.038161 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":265,"threads_started":1}
I20260806 01:00:26.039837 6103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:26.020972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:42087 (local address 127.5.108.130:44645)
0806 01:00:26.021683 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:26.021725 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:26.023454 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:26.024146 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:26.024155 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:26.024709 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:26.025247 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:26.025264 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.026342 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.026352 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:26.027826 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:26.027840 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.028698 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.028708 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:26.028941 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:26.029624 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:26.029650 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:26.032324 (+ 2674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:26.035409 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:26.035418 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:26.035433 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:26.036023 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:26.036361 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:26.036367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:26.036373 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:26.036643 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:26.037120 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:26.037130 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:26.037556 (+ 426us) client_negotiation.cc:769] Sending connection context
0806 01:00:26.037860 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 01:00:26.038408 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":305,"threads_started":1}
I20260806 01:00:26.041311 6097 heartbeater.cc:347] Connected to a master server at 127.5.108.190:42087
I20260806 01:00:26.041622 6097 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:26.042297 6097 heartbeater.cc:511] Master 127.5.108.190:42087 requested a full tablet report, sending...
I20260806 01:00:26.043778 5621 ts_manager.cc:194] Registered new tserver with Master: b745980ca96c42ceaa0a86966a99d197 (127.5.108.130:40479)
I20260806 01:00:26.045017 5621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:44645
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:00:26.068392 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42087
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:26.391130 6106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:26.391445 6106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:26.391561 6106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:26.398149 6106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:26.398237 6106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:26.398339 6106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:00:26.407625 6106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:26.408715 6106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:26.409888 6106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:26.422792 6111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:26.422830 6112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:26.422854 6114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:26.424088 6106 server_base.cc:1034] running on GCE node
I20260806 01:00:26.424628 6106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:26.425457 6106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:26.426692 6106 hybrid_clock.cc:648] HybridClock initialized: now 1785978026426643 us; error 58 us; skew 500 ppm
I20260806 01:00:26.426988 6106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:26 dist-test-slave-x4qh krb5kdc[5561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978026, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:26.429943 6106 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:00:26.431293 6106 webserver.cc:460] Webserver started at http://127.5.108.131:41519/ using document root <none> and password file <none>
I20260806 01:00:26.431957 6106 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:26.432107 6106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:26.432441 6106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:26.435006 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "9b86534ff2c64251900ccec4bd33d6e6"
format_stamp: "Formatted at 2026-08-06 01:00:26 on dist-test-slave-x4qh"
server_key: "f138cafe3244c700d126d7dce09e56d9"
server_key_iv: "f3a38e9b3f776276c17f072319e20780"
server_key_version: "encryptionkey@0"
I20260806 01:00:26.435751 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "9b86534ff2c64251900ccec4bd33d6e6"
format_stamp: "Formatted at 2026-08-06 01:00:26 on dist-test-slave-x4qh"
server_key: "f138cafe3244c700d126d7dce09e56d9"
server_key_iv: "f3a38e9b3f776276c17f072319e20780"
server_key_version: "encryptionkey@0"
I20260806 01:00:26.440713 6106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 01:00:26.443852 6124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:26.444942 6106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260806 01:00:26.445112 6106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "9b86534ff2c64251900ccec4bd33d6e6"
format_stamp: "Formatted at 2026-08-06 01:00:26 on dist-test-slave-x4qh"
server_key: "f138cafe3244c700d126d7dce09e56d9"
server_key_iv: "f3a38e9b3f776276c17f072319e20780"
server_key_version: "encryptionkey@0"
I20260806 01:00:26.445341 6106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:26.462981 6106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:26.466991 6106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:26.467283 6106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:26.468180 6106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:26.469484 6106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:26.469584 6106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:26.469728 6106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:26.469797 6106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:26.495705 5902 heartbeater.cc:503] Master 127.5.108.190:42087 was elected leader, sending a full tablet report...
I20260806 01:00:26.506247 6106 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:38493
I20260806 01:00:26.506287 6280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:38493 every 8 connection(s)
I20260806 01:00:26.507575 6106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:00:26.513645 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6106
I20260806 01:00:26.513837 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:00:26.514266 5554 external_mini_cluster.cc:1428] Setting key db12e0d4186eed2afb0cfdf6cab47cf3
Aug 06 01:00:26 dist-test-slave-x4qh krb5kdc[5561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978026, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:26.526774 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:26.509978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:38797 (local address 127.5.108.190:42087)
0806 01:00:26.510157 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:26.510162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:26.511364 (+ 1202us) server_negotiation.cc:407] Connection header received
0806 01:00:26.513238 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:26.513244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:26.513330 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:26.513453 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:26.515088 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.515954 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.517445 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.517857 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.518008 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:26.521457 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:26.521467 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:26.521470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:26.521501 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:26.523895 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.524614 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.524622 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.524627 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.524761 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.525175 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.525181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.525185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.525448 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:26.525624 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:26.526248 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 01:00:26.526454 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:00:26.527722 6284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:26.510466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:42087 (local address 127.5.108.131:38797)
0806 01:00:26.511193 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:26.511229 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:26.512969 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:26.513696 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:26.513704 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:26.514341 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:26.514913 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:26.514930 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.516114 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.516119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:26.517295 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:26.517307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.518109 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.518117 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:26.518347 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:26.519022 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:26.519049 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:26.521259 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:26.524031 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:26.524039 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:26.524053 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:26.524492 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:26.524883 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:26.524888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:26.524891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:26.525062 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:26.525581 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:26.525589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:26.525899 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 01:00:26.526166 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 01:00:26.526619 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":277,"threads_started":1}
I20260806 01:00:26.529000 6282 heartbeater.cc:347] Connected to a master server at 127.5.108.190:42087
I20260806 01:00:26.529328 6282 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:26.530074 6282 heartbeater.cc:511] Master 127.5.108.190:42087 requested a full tablet report, sending...
I20260806 01:00:26.531440 5621 ts_manager.cc:194] Registered new tserver with Master: 9b86534ff2c64251900ccec4bd33d6e6 (127.5.108.131:38493)
I20260806 01:00:26.532266 5621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:38797
I20260806 01:00:26.542449 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:00:26 dist-test-slave-x4qh krb5kdc[5561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:00:26 dist-test-slave-x4qh krb5kdc[5561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:00:26.591670 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:26.576360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43348 (local address 127.5.108.129:46725)
0806 01:00:26.577039 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:26.577050 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:26.577118 (+ 68us) server_negotiation.cc:407] Connection header received
0806 01:00:26.577359 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:26.577367 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:26.577576 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:26.577756 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:26.579257 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.580337 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.581652 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.582225 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.582364 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:26.586249 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:26.586260 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:26.586272 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:26.586308 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:26.588978 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.589610 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.589616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.589618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.589746 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.590095 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.590100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.590104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.590415 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:26.590759 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:26.591028 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 01:00:26.591288 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":316,"threads_started":1}
W20260806 01:00:26.592960 5777 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43348
Aug 06 01:00:26 dist-test-slave-x4qh krb5kdc[5561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:00:26 dist-test-slave-x4qh krb5kdc[5561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:00:26.639282 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:26.626038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.5.108.129:46725)
0806 01:00:26.626249 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:26.626255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:26.626470 (+ 215us) server_negotiation.cc:407] Connection header received
0806 01:00:26.626664 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:26.626668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:26.626726 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:26.626838 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:26.628233 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.628916 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.630103 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.630533 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.630669 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:26.634402 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:26.634407 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:26.634410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:26.634440 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:26.636791 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.637351 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.637357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.637361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.637482 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.637817 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.637822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.637825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.638162 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:26.638439 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:26.638720 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 01:00:26.638949 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:00:26.641148 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5719
I20260806 01:00:26.653584 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:26.642075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.5.108.129:46725)
0806 01:00:26.642272 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:26.642279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:26.642556 (+ 277us) server_negotiation.cc:407] Connection header received
0806 01:00:26.642741 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:26.642745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:26.642794 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:26.642894 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:26.644119 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.644778 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.646020 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.646416 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.646546 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:26.648753 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:26.648758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:26.648761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:26.648788 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:26.651101 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.651805 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.651811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.651814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.651948 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.652338 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.652342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.652345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.652568 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:26.652793 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:26.653085 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 01:00:26.653266 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 01:00:26.654249 5896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:26.738324 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5719
I20260806 01:00:26.765118 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5908
Aug 06 01:00:26 dist-test-slave-x4qh krb5kdc[5561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:00:26.780778 6303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:26.766041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.5.108.130:40479)
0806 01:00:26.766619 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:26.766626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:26.766653 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:00:26.766740 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:26.766747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:26.766960 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:26.767095 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:26.768278 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.769270 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.770498 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.771053 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.771154 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:26.775157 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:26.775168 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:26.775181 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:26.775212 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:26.778285 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.778897 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.778905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.778907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.779025 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.779331 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.779334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.779336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.779727 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:26.779968 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:26.780253 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 01:00:26.780437 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":304,"threads_started":1}
I20260806 01:00:26.782044 6092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:26.864931 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5908
I20260806 01:00:26.889371 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6106
Aug 06 01:00:26 dist-test-slave-x4qh krb5kdc[5561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:00:26.905910 6306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:26.890692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.5.108.131:38493)
0806 01:00:26.891418 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:26.891429 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:26.891464 (+ 35us) server_negotiation.cc:407] Connection header received
0806 01:00:26.891540 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:26.891547 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:26.891772 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:26.891901 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:26.893081 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.894059 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.895816 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:26.896344 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:26.896435 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:26.900621 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:26.900632 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:26.900646 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:26.900682 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:26.903257 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.903997 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.904003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.904006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.904123 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:26.904458 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:26.904462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:26.904464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:26.904801 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:26.905044 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:26.905329 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 01:00:26.905522 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":358,"threads_started":1}
I20260806 01:00:26.907186 6275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:26.990859 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6106
I20260806 01:00:27.013530 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5576
I20260806 01:00:27.014724 5671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:27.114992 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5576
2026-08-06T01:00:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4299 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:00:27 dist-test-slave-x4qh krb5kdc[6317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:00:27 dist-test-slave-x4qh krb5kdc[6317](info): set up 2 sockets
Aug 06 01:00:27 dist-test-slave-x4qh krb5kdc[6317](info): commencing operation
krb5kdc: starting...
W20260806 01:00:28.355962 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:00:28 dist-test-slave-x4qh krb5kdc[6317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:00:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:00:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:00:28.524367 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:35253
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:35253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:28.832998 6337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:28.833227 6337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:28.833387 6337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:28.840037 6337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:00:28.840126 6337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:28.840225 6337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:28.840334 6337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:00:28.840406 6337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:00:28.849478 6337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:35253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:35253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:28.850561 6337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:28.851876 6337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:28.862718 6348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:28.862764 6344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:28.862963 6351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:28.863502 6337 server_base.cc:1034] running on GCE node
I20260806 01:00:28.864192 6337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:28.865419 6337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:28.866622 6337 hybrid_clock.cc:648] HybridClock initialized: now 1785978028866582 us; error 30 us; skew 500 ppm
I20260806 01:00:28.866919 6337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:28 dist-test-slave-x4qh krb5kdc[6317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978028, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:28.873158 6337 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:00:28.874550 6337 webserver.cc:460] Webserver started at http://127.5.108.190:41445/ using document root <none> and password file <none>
I20260806 01:00:28.875247 6337 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:28.875433 6337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:28.875754 6337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:28.878551 6337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "825828b9d68341b4b317153149ff9278"
format_stamp: "Formatted at 2026-08-06 01:00:28 on dist-test-slave-x4qh"
server_key: "114f9542fde871263504f3c1e2171663"
server_key_iv: "eed8cf015e3425a0042acdd868535df2"
server_key_version: "encryptionkey@0"
I20260806 01:00:28.879245 6337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "825828b9d68341b4b317153149ff9278"
format_stamp: "Formatted at 2026-08-06 01:00:28 on dist-test-slave-x4qh"
server_key: "114f9542fde871263504f3c1e2171663"
server_key_iv: "eed8cf015e3425a0042acdd868535df2"
server_key_version: "encryptionkey@0"
I20260806 01:00:28.884124 6337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 01:00:28.887229 6361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:28.888595 6337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:00:28.888793 6337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "825828b9d68341b4b317153149ff9278"
format_stamp: "Formatted at 2026-08-06 01:00:28 on dist-test-slave-x4qh"
server_key: "114f9542fde871263504f3c1e2171663"
server_key_iv: "eed8cf015e3425a0042acdd868535df2"
server_key_version: "encryptionkey@0"
I20260806 01:00:28.888979 6337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:28.902980 6337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:28.906618 6337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:28.906868 6337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:28.924608 6337 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:35253
I20260806 01:00:28.924626 6432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:35253 every 8 connection(s)
I20260806 01:00:28.925930 6337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:00:28.928359 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6337
I20260806 01:00:28.928583 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:00:28.929212 5554 external_mini_cluster.cc:1428] Setting key 3b65bf68d7c25b0c1f2ed9ebc83d3c49
I20260806 01:00:28.929379 6433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:28.942750 6433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278: Bootstrap starting.
Aug 06 01:00:28 dist-test-slave-x4qh krb5kdc[6317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:28.946432 6433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:28.947759 6433 log.cc:824] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:28.951045 6433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278: No bootstrap required, opened a new log
I20260806 01:00:28.952587 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:28.931066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57984 (local address 127.5.108.190:35253)
0806 01:00:28.931869 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:28.931879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:28.931939 (+ 60us) server_negotiation.cc:407] Connection header received
0806 01:00:28.933681 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:28.933699 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:28.933999 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:28.934306 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:28.935930 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:28.937048 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:28.938677 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:28.939330 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:28.939530 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:28.944038 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:28.944050 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:28.944070 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:28.944126 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:28.948047 (+ 3921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:28.948937 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:28.948942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:28.948947 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:28.949148 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:28.949542 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:28.949547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:28.949550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:28.950216 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:28.950504 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:28.950897 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 01:00:28.951285 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":403,"threads_started":1}
I20260806 01:00:28.958480 6433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825828b9d68341b4b317153149ff9278" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35253 } }
I20260806 01:00:28.958774 6433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:28.958845 6433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825828b9d68341b4b317153149ff9278, State: Initialized, Role: FOLLOWER
I20260806 01:00:28.959280 6433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825828b9d68341b4b317153149ff9278" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35253 } }
I20260806 01:00:28.959544 6433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:00:28.959640 6433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:00:28.959750 6433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:28.961654 6433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825828b9d68341b4b317153149ff9278" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35253 } }
I20260806 01:00:28.962065 6433 leader_election.cc:302] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 825828b9d68341b4b317153149ff9278; no voters:
I20260806 01:00:28.962523 6433 leader_election.cc:288] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:00:28.962677 6439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:28.963948 6439 raft_consensus.cc:686] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [term 1 LEADER]: Becoming Leader. State: Replica: 825828b9d68341b4b317153149ff9278, State: Running, Role: LEADER
I20260806 01:00:28.964480 6439 consensus_queue.cc:237] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825828b9d68341b4b317153149ff9278" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35253 } }
I20260806 01:00:28.964617 6433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:00:28.966741 6442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 825828b9d68341b4b317153149ff9278. Latest consensus state: current_term: 1 leader_uuid: "825828b9d68341b4b317153149ff9278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825828b9d68341b4b317153149ff9278" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35253 } } }
I20260806 01:00:28.967044 6442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:28.967677 6440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "825828b9d68341b4b317153149ff9278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825828b9d68341b4b317153149ff9278" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35253 } } }
I20260806 01:00:28.967979 6440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278 [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:28.968003 6450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:00:28.976315 6450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:00:28.985195 6450 catalog_manager.cc:1347] Generated new cluster ID: 76206426b03a4058a8a2a5436e0d405e
I20260806 01:00:28.985337 6450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:00:28.998072 6450 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:00:28.999267 6450 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:00:29.009443 6450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 825828b9d68341b4b317153149ff9278: Generated new TSK 0
I20260806 01:00:29.010334 6450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:00:29.052920 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35253
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:29.360581 6471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:29.360867 6471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:29.361037 6471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:29.368088 6471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:29.368184 6471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:29.368341 6471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:00:29.377812 6471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35253
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:29.379017 6471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:29.380293 6471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:29.393301 6477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:29.393244 6480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:29.393277 6484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:29.393913 6471 server_base.cc:1034] running on GCE node
I20260806 01:00:29.394403 6471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:29.395251 6471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:29.396435 6471 hybrid_clock.cc:648] HybridClock initialized: now 1785978029396396 us; error 48 us; skew 500 ppm
I20260806 01:00:29.396697 6471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:29 dist-test-slave-x4qh krb5kdc[6317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978029, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:29.399564 6471 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:00:29.400966 6471 webserver.cc:460] Webserver started at http://127.5.108.129:36245/ using document root <none> and password file <none>
I20260806 01:00:29.401600 6471 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:29.401751 6471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:29.402089 6471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:29.404829 6471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "ae7f466276b84fa48b7848680b9b592c"
format_stamp: "Formatted at 2026-08-06 01:00:29 on dist-test-slave-x4qh"
server_key: "add52fdf6d4e123af9e2e4001e712b04"
server_key_iv: "189fd16e43b003f3f695609bb0b7822f"
server_key_version: "encryptionkey@0"
I20260806 01:00:29.405580 6471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "ae7f466276b84fa48b7848680b9b592c"
format_stamp: "Formatted at 2026-08-06 01:00:29 on dist-test-slave-x4qh"
server_key: "add52fdf6d4e123af9e2e4001e712b04"
server_key_iv: "189fd16e43b003f3f695609bb0b7822f"
server_key_version: "encryptionkey@0"
I20260806 01:00:29.410419 6471 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 01:00:29.413679 6494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:29.414824 6471 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:00:29.414989 6471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "ae7f466276b84fa48b7848680b9b592c"
format_stamp: "Formatted at 2026-08-06 01:00:29 on dist-test-slave-x4qh"
server_key: "add52fdf6d4e123af9e2e4001e712b04"
server_key_iv: "189fd16e43b003f3f695609bb0b7822f"
server_key_version: "encryptionkey@0"
I20260806 01:00:29.415175 6471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:29.428323 6471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:29.431790 6471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:29.432072 6471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:29.432878 6471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:29.434031 6471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:29.434151 6471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:29.434283 6471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:29.434355 6471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:29.466107 6471 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:34211
I20260806 01:00:29.466176 6649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:34211 every 8 connection(s)
I20260806 01:00:29.467199 6471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:00:29.468284 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6471
I20260806 01:00:29.468520 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:00:29.469316 5554 external_mini_cluster.cc:1428] Setting key 87ff05f547643810d3c8ce2a345b012e
Aug 06 01:00:29 dist-test-slave-x4qh krb5kdc[6317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978029, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:29.487143 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:29.469853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:40745 (local address 127.5.108.190:35253)
0806 01:00:29.470321 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:29.470327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:29.471306 (+ 979us) server_negotiation.cc:407] Connection header received
0806 01:00:29.473293 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:29.473298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:29.473373 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:29.473509 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:29.474810 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:29.475639 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:29.477094 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:29.477500 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:29.477640 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:29.481433 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:29.481438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:29.481441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:29.481472 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:29.484100 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:29.484890 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:29.484896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:29.484899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:29.485024 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:29.485440 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:29.485446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:29.485450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:29.485705 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:29.485866 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:29.486545 (+ 679us) server_negotiation.cc:299] Negotiation successful
0806 01:00:29.486772 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":240,"threads_started":1}
I20260806 01:00:29.488071 6657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:29.470339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:35253 (local address 127.5.108.129:40745)
0806 01:00:29.471101 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:29.471187 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:29.473046 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:29.473728 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:29.473735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:29.474172 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:29.474653 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:29.474669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:29.475807 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:29.475817 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:29.476931 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:29.476967 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:29.477735 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:29.477744 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:29.477965 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:29.478674 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:29.478704 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:29.481217 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:29.484259 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:29.484267 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:29.484279 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:29.484773 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:29.485137 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:29.485142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:29.485146 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:29.485328 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:29.485843 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:29.485850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:29.486146 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 01:00:29.486469 (+ 323us) client_negotiation.cc:240] Negotiation successful
0806 01:00:29.486934 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":285,"threads_started":1}
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
I20260806 01:00:29.489456 6651 heartbeater.cc:347] Connected to a master server at 127.5.108.190:35253
I20260806 01:00:29.489710 6651 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:29.490360 6651 heartbeater.cc:511] Master 127.5.108.190:35253 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
I20260806 01:00:29.492609 6385 ts_manager.cc:194] Registered new tserver with Master: ae7f466276b84fa48b7848680b9b592c (127.5.108.129:34211)
I20260806 01:00:29.494068 6385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:40745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:00:29.512336 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35253
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:29.826491 6660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:29.826761 6660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:29.826968 6660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:29.834870 6660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:29.835016 6660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:29.835214 6660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:00:29.844746 6660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35253
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:29.845896 6660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:29.847177 6660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:29.860020 6674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:29.860069 6670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:29.860095 6677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:29.861204 6660 server_base.cc:1034] running on GCE node
I20260806 01:00:29.861799 6660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:29.862659 6660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:29.863855 6660 hybrid_clock.cc:648] HybridClock initialized: now 1785978029863802 us; error 50 us; skew 500 ppm
I20260806 01:00:29.864147 6660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:29 dist-test-slave-x4qh krb5kdc[6317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978029, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:29.867113 6660 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:00:29.868714 6660 webserver.cc:460] Webserver started at http://127.5.108.130:42327/ using document root <none> and password file <none>
I20260806 01:00:29.869361 6660 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:29.869522 6660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:29.869840 6660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:29.872712 6660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "1d56cefc3b0b414b9e4513c65c152c42"
format_stamp: "Formatted at 2026-08-06 01:00:29 on dist-test-slave-x4qh"
server_key: "102d04c752eda3314aa8e255a3cfa73b"
server_key_iv: "6c1a4119c70cb0d88b5bd776b517d53b"
server_key_version: "encryptionkey@0"
I20260806 01:00:29.873499 6660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "1d56cefc3b0b414b9e4513c65c152c42"
format_stamp: "Formatted at 2026-08-06 01:00:29 on dist-test-slave-x4qh"
server_key: "102d04c752eda3314aa8e255a3cfa73b"
server_key_iv: "6c1a4119c70cb0d88b5bd776b517d53b"
server_key_version: "encryptionkey@0"
I20260806 01:00:29.878788 6660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 01:00:29.882311 6685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:29.883719 6660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:00:29.883899 6660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "1d56cefc3b0b414b9e4513c65c152c42"
format_stamp: "Formatted at 2026-08-06 01:00:29 on dist-test-slave-x4qh"
server_key: "102d04c752eda3314aa8e255a3cfa73b"
server_key_iv: "6c1a4119c70cb0d88b5bd776b517d53b"
server_key_version: "encryptionkey@0"
I20260806 01:00:29.884078 6660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:29.900331 6660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:29.904520 6660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:29.904803 6660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:29.905714 6660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:29.906988 6660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:29.907060 6660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:29.907236 6660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:29.907287 6660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:29.941058 6660 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:44887
I20260806 01:00:29.941094 6846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:44887 every 8 connection(s)
I20260806 01:00:29.942339 6660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:00:29.948215 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6660
I20260806 01:00:29.948418 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:00:29.948954 5554 external_mini_cluster.cc:1428] Setting key 3a072eed78c7891b6082c87f89e58d11
Aug 06 01:00:29 dist-test-slave-x4qh krb5kdc[6317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978029, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:29.961246 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:29.944722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:49057 (local address 127.5.108.190:35253)
0806 01:00:29.944893 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:29.944898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:29.946006 (+ 1108us) server_negotiation.cc:407] Connection header received
0806 01:00:29.947825 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:29.947830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:29.947919 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:29.948045 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:29.949787 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:29.950623 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:29.952303 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:29.952746 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:29.952880 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:29.956102 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:29.956107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:29.956109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:29.956141 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:29.958414 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:29.959087 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:29.959093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:29.959101 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:29.959248 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:29.959701 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:29.959705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:29.959707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:29.959935 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:29.960093 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:29.960682 (+ 589us) server_negotiation.cc:299] Negotiation successful
0806 01:00:29.960925 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:00:29.962107 6849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:29.945164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:35253 (local address 127.5.108.130:49057)
0806 01:00:29.945829 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:29.945877 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:29.947583 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:29.948304 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:29.948316 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:29.948921 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:29.949566 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:29.949592 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:29.950792 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:29.950797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:29.952159 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:29.952171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:29.952970 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:29.952977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:29.953200 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:29.953866 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:29.953892 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:29.955900 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:29.958540 (+ 2640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:29.958548 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:29.958561 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:29.958949 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:29.959387 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:29.959391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:29.959395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:29.959576 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:29.960060 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:29.960067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:29.960359 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 01:00:29.960589 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 01:00:29.961051 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":268,"threads_started":1}
I20260806 01:00:29.963472 6847 heartbeater.cc:347] Connected to a master server at 127.5.108.190:35253
I20260806 01:00:29.963771 6847 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:29.964530 6847 heartbeater.cc:511] Master 127.5.108.190:35253 requested a full tablet report, sending...
I20260806 01:00:29.965983 6385 ts_manager.cc:194] Registered new tserver with Master: 1d56cefc3b0b414b9e4513c65c152c42 (127.5.108.130:44887)
I20260806 01:00:29.966773 6385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:49057
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:00:29.988870 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35253
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:30.297263 6853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:30.297556 6853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:30.297726 6853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:30.304828 6853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:30.304957 6853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:30.305150 6853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:00:30.314054 6853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35253
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:30.315110 6853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:30.316437 6853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:30.329237 6860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:30.329304 6862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:30.329334 6864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:30.329986 6853 server_base.cc:1034] running on GCE node
I20260806 01:00:30.330538 6853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:30.331480 6853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:30.332722 6853 hybrid_clock.cc:648] HybridClock initialized: now 1785978030332679 us; error 38 us; skew 500 ppm
I20260806 01:00:30.333030 6853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:30 dist-test-slave-x4qh krb5kdc[6317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978030, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:30.336115 6853 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:00:30.337492 6853 webserver.cc:460] Webserver started at http://127.5.108.131:45777/ using document root <none> and password file <none>
I20260806 01:00:30.338172 6853 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:30.338321 6853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:30.338646 6853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:30.341436 6853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "cb47395a642945119c57029b3eab8d67"
format_stamp: "Formatted at 2026-08-06 01:00:30 on dist-test-slave-x4qh"
server_key: "83ab401e863e924f2c7189da4f19a5d1"
server_key_iv: "0c5fcf4c463d2b24c989389407bf9e1d"
server_key_version: "encryptionkey@0"
I20260806 01:00:30.342222 6853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "cb47395a642945119c57029b3eab8d67"
format_stamp: "Formatted at 2026-08-06 01:00:30 on dist-test-slave-x4qh"
server_key: "83ab401e863e924f2c7189da4f19a5d1"
server_key_iv: "0c5fcf4c463d2b24c989389407bf9e1d"
server_key_version: "encryptionkey@0"
I20260806 01:00:30.347035 6853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:00:30.350337 6875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:30.351617 6853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:00:30.351794 6853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "cb47395a642945119c57029b3eab8d67"
format_stamp: "Formatted at 2026-08-06 01:00:30 on dist-test-slave-x4qh"
server_key: "83ab401e863e924f2c7189da4f19a5d1"
server_key_iv: "0c5fcf4c463d2b24c989389407bf9e1d"
server_key_version: "encryptionkey@0"
I20260806 01:00:30.352000 6853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:30.369521 6853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:30.373246 6853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:30.373518 6853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:30.374450 6853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:30.375756 6853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:30.375857 6853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:30.376003 6853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:30.376076 6853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:30.409497 6853 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:40843
I20260806 01:00:30.409582 7042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:40843 every 8 connection(s)
I20260806 01:00:30.410713 6853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:00:30.414000 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6853
I20260806 01:00:30.414254 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:00:30.414791 5554 external_mini_cluster.cc:1428] Setting key a9816a34ac14b865065ba3f065338ffb
Aug 06 01:00:30 dist-test-slave-x4qh krb5kdc[6317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978030, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:30.430408 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.413181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:37565 (local address 127.5.108.190:35253)
0806 01:00:30.413356 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:30.413364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:30.414864 (+ 1500us) server_negotiation.cc:407] Connection header received
0806 01:00:30.417145 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:30.417148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:30.417207 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:30.417288 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:30.418846 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.419569 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.421073 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.421493 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.421586 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.424768 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:30.424772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:30.424775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:30.424805 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:30.427435 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:30.428072 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:30.428077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:30.428080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:30.428227 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:30.428723 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:30.428730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:30.428735 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:30.429085 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:30.429211 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:30.430013 (+ 802us) server_negotiation.cc:299] Negotiation successful
0806 01:00:30.430184 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 01:00:30.431202 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.413804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:35253 (local address 127.5.108.131:37565)
0806 01:00:30.414700 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:30.414745 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:30.416884 (+ 2139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:30.417546 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:30.417556 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:30.418128 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:30.418695 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.418729 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.419709 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.419713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.420953 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.420965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.421693 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.421698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.421873 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.422436 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:30.422461 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:30.424577 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:30.427542 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:30.427547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:30.427567 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:30.427964 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:30.428356 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:30.428362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:30.428367 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:30.428574 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:30.429223 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:30.429233 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:30.429586 (+ 353us) client_negotiation.cc:769] Sending connection context
0806 01:00:30.429910 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 01:00:30.430254 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":384,"threads_started":1}
I20260806 01:00:30.432490 7043 heartbeater.cc:347] Connected to a master server at 127.5.108.190:35253
I20260806 01:00:30.432766 7043 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:30.433418 7043 heartbeater.cc:511] Master 127.5.108.190:35253 requested a full tablet report, sending...
I20260806 01:00:30.434643 6385 ts_manager.cc:194] Registered new tserver with Master: cb47395a642945119c57029b3eab8d67 (127.5.108.131:40843)
I20260806 01:00:30.435433 6385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:37565
I20260806 01:00:30.441679 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:00:30.459736 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.448487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.5.108.190:35253)
0806 01:00:30.448694 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:30.448702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:30.449065 (+ 363us) server_negotiation.cc:407] Connection header received
0806 01:00:30.449164 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:30.449167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:30.449217 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:30.449291 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:30.450669 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.451419 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.452628 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.453025 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.453112 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.455115 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:30.455120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:30.455126 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:30.455158 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:30.457604 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:30.458121 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:30.458124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:30.458126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:30.458245 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:30.458560 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:30.458562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:30.458564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:30.458823 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:30.458990 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:30.459230 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 01:00:30.459414 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:00:30.467231 6385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:00:30.475028 6385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:00:30.497413 6651 heartbeater.cc:503] Master 127.5.108.190:35253 was elected leader, sending a full tablet report...
I20260806 01:00:30.503574 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.489992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44887 (local address 127.0.0.1:42862)
0806 01:00:30.490992 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:30.491008 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:30.491116 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:30.496247 (+ 5131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:30.496253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:30.496279 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:30.496674 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.496686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.498624 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.498630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.500763 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.500774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.502633 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.502640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.502854 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.502904 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 01:00:30.503012 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 01:00:30.503188 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":6950,"threads_started":1}
I20260806 01:00:30.504638 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.491495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.5.108.130:44887)
0806 01:00:30.495789 (+ 4294us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:30.495794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:30.495814 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:00:30.495889 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:30.495894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:30.496037 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:30.496232 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:30.496878 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.498454 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.500944 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.502428 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.503831 (+ 1403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.504004 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:30.504095 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 01:00:30.504266 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4170,"thread_start_us":4050,"threads_started":1}
I20260806 01:00:30.505115 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.491065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.5.108.129:34211)
0806 01:00:30.491969 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:30.491977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:30.497309 (+ 5332us) server_negotiation.cc:407] Connection header received
0806 01:00:30.497608 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:30.497615 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:30.497854 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:30.498023 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:30.498997 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.500610 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.502573 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.504008 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.504171 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.504413 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:30.504631 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 01:00:30.504788 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":644,"threads_started":1}
I20260806 01:00:30.505383 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.489330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:40843 (local address 127.0.0.1:34482)
0806 01:00:30.490068 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:30.490116 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:30.490279 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:30.497461 (+ 7182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:30.497466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:30.497490 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:30.497905 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.497929 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.500092 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.500097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.502402 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.502417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.504566 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.504574 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.504757 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.504790 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:00:30.504867 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 01:00:30.504995 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":38,"thread_start_us":439,"threads_started":1}
I20260806 01:00:30.505949 7064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.490473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:34211 (local address 127.0.0.1:42280)
0806 01:00:30.497209 (+ 6736us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:30.497226 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:30.497420 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:30.498318 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:30.498321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:30.498343 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:30.498798 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.498812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.500761 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.500766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.502365 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.502377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.504220 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.504226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.504403 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.504438 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 01:00:30.504588 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 01:00:30.504706 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6601,"mutex_wait_us":21,"thread_start_us":1405,"threads_started":1}
I20260806 01:00:30.506301 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.489951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34482 (local address 127.5.108.131:40843)
0806 01:00:30.496908 (+ 6957us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:30.496915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:30.496944 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:00:30.497035 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:30.497041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:30.497230 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:30.498112 (+ 882us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:30.498342 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.499953 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.502595 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.504310 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.505040 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.505532 (+ 492us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:30.505728 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 01:00:30.505939 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6799,"thread_start_us":6649,"threads_started":1}
I20260806 01:00:30.509115 6946 tablet_service.cc:1467] Processing CreateTablet for tablet 41bfb493e52946cabd1828c25774b8d1 (DEFAULT_TABLE table=test-table [id=b47d942bef2c49cc8188a9bfdbc8b13c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:30.508999 6753 tablet_service.cc:1467] Processing CreateTablet for tablet 41bfb493e52946cabd1828c25774b8d1 (DEFAULT_TABLE table=test-table [id=b47d942bef2c49cc8188a9bfdbc8b13c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:30.509704 6559 tablet_service.cc:1467] Processing CreateTablet for tablet 41bfb493e52946cabd1828c25774b8d1 (DEFAULT_TABLE table=test-table [id=b47d942bef2c49cc8188a9bfdbc8b13c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:30.510476 6946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41bfb493e52946cabd1828c25774b8d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:30.510502 6753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41bfb493e52946cabd1828c25774b8d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:30.510982 6559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41bfb493e52946cabd1828c25774b8d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:30.521770 7077 tablet_bootstrap.cc:492] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67: Bootstrap starting.
I20260806 01:00:30.525094 7078 tablet_bootstrap.cc:492] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42: Bootstrap starting.
I20260806 01:00:30.526494 7077 tablet_bootstrap.cc:654] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:30.527788 7078 tablet_bootstrap.cc:654] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:30.527995 7077 log.cc:824] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:30.528793 7078 log.cc:824] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:30.529345 7079 tablet_bootstrap.cc:492] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c: Bootstrap starting.
I20260806 01:00:30.531086 7077 tablet_bootstrap.cc:492] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67: No bootstrap required, opened a new log
I20260806 01:00:30.531352 7077 ts_tablet_manager.cc:1397] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260806 01:00:30.532341 7078 tablet_bootstrap.cc:492] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42: No bootstrap required, opened a new log
I20260806 01:00:30.532599 7078 ts_tablet_manager.cc:1397] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 01:00:30.533149 7079 tablet_bootstrap.cc:654] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:30.534305 7079 log.cc:824] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:30.537065 7079 tablet_bootstrap.cc:492] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c: No bootstrap required, opened a new log
I20260806 01:00:30.537297 7079 ts_tablet_manager.cc:1397] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 01:00:30.538825 7077 raft_consensus.cc:348] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } }
I20260806 01:00:30.539155 7077 raft_consensus.cc:374] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:30.539248 7077 raft_consensus.cc:729] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb47395a642945119c57029b3eab8d67, State: Initialized, Role: FOLLOWER
I20260806 01:00:30.539803 7077 consensus_queue.cc:260] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } }
I20260806 01:00:30.540951 7043 heartbeater.cc:503] Master 127.5.108.190:35253 was elected leader, sending a full tablet report...
I20260806 01:00:30.541571 7077 ts_tablet_manager.cc:1428] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67: Time spent starting tablet: real 0.010s user 0.007s sys 0.002s
I20260806 01:00:30.542268 7078 raft_consensus.cc:348] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } }
I20260806 01:00:30.542641 7078 raft_consensus.cc:374] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:30.542739 7078 raft_consensus.cc:729] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d56cefc3b0b414b9e4513c65c152c42, State: Initialized, Role: FOLLOWER
I20260806 01:00:30.543260 7078 consensus_queue.cc:260] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } }
I20260806 01:00:30.545202 6847 heartbeater.cc:503] Master 127.5.108.190:35253 was elected leader, sending a full tablet report...
I20260806 01:00:30.545655 7078 ts_tablet_manager.cc:1428] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260806 01:00:30.547524 7079 raft_consensus.cc:348] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } }
I20260806 01:00:30.547922 7079 raft_consensus.cc:374] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:30.548022 7079 raft_consensus.cc:729] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae7f466276b84fa48b7848680b9b592c, State: Initialized, Role: FOLLOWER
I20260806 01:00:30.548544 7079 consensus_queue.cc:260] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } }
I20260806 01:00:30.550079 7079 ts_tablet_manager.cc:1428] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
W20260806 01:00:30.662355 7046 tablet.cc:2369] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:00:30.694146 6848 tablet.cc:2369] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:00:30.719519 6653 tablet.cc:2369] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:00:30.894378 7086 raft_consensus.cc:482] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:30.894728 7086 raft_consensus.cc:504] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } }
I20260806 01:00:30.896991 7086 leader_election.cc:288] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb47395a642945119c57029b3eab8d67 (127.5.108.131:40843), 1d56cefc3b0b414b9e4513c65c152c42 (127.5.108.130:44887)
I20260806 01:00:30.903666 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.897446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:56271 (local address 127.5.108.130:44887)
0806 01:00:30.897687 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:30.897693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:30.898382 (+ 689us) server_negotiation.cc:407] Connection header received
0806 01:00:30.898626 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:30.898638 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:30.898704 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:30.898803 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:30.899418 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.900133 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.901923 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.902699 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.902880 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.902958 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:30.903239 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 01:00:30.903362 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:00:30.903791 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.897505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44887 (local address 127.5.108.129:56271)
0806 01:00:30.898260 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:30.898279 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:30.898466 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:30.898824 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:30.898828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:30.898857 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:30.899245 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.899255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.900263 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.900271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.901753 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.901766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.902914 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.902919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.903070 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.903094 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:00:30.903209 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 01:00:30.903340 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":3,"spinlock_wait_cycles":35072,"thread_start_us":425,"threads_started":1}
I20260806 01:00:30.904042 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.897531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:35555 (local address 127.5.108.131:40843)
0806 01:00:30.897714 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:30.897722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:30.898836 (+ 1114us) server_negotiation.cc:407] Connection header received
0806 01:00:30.899112 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:30.899121 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:30.899179 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:30.899282 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:30.899836 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.900515 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.902334 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.903097 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.903291 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.903364 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:30.903690 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 01:00:30.903802 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:00:30.904199 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.897862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:40843 (local address 127.5.108.129:35555)
0806 01:00:30.898714 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:30.898728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:30.898950 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:30.899303 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:30.899307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:30.899330 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:30.899665 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.899674 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.900670 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.900677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.902169 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:30.902184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.903322 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.903328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:30.903499 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.903523 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:00:30.903633 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 01:00:30.903766 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":25,"spinlock_wait_cycles":24320,"thread_start_us":506,"threads_started":1}
I20260806 01:00:30.905270 6981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41bfb493e52946cabd1828c25774b8d1" candidate_uuid: "ae7f466276b84fa48b7848680b9b592c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb47395a642945119c57029b3eab8d67" is_pre_election: true
I20260806 01:00:30.905272 6783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41bfb493e52946cabd1828c25774b8d1" candidate_uuid: "ae7f466276b84fa48b7848680b9b592c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d56cefc3b0b414b9e4513c65c152c42" is_pre_election: true
I20260806 01:00:30.905967 6783 raft_consensus.cc:2454] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae7f466276b84fa48b7848680b9b592c in term 0.
I20260806 01:00:30.906693 6497 leader_election.cc:302] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d56cefc3b0b414b9e4513c65c152c42, ae7f466276b84fa48b7848680b9b592c; no voters:
I20260806 01:00:30.907125 7086 raft_consensus.cc:2781] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:00:30.907258 7086 raft_consensus.cc:482] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:00:30.907334 7086 raft_consensus.cc:3037] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:30.907920 6981 raft_consensus.cc:2454] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae7f466276b84fa48b7848680b9b592c in term 0.
I20260806 01:00:30.909795 7086 raft_consensus.cc:504] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } }
I20260806 01:00:30.910344 7086 leader_election.cc:288] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [CANDIDATE]: Term 1 election: Requested vote from peers cb47395a642945119c57029b3eab8d67 (127.5.108.131:40843), 1d56cefc3b0b414b9e4513c65c152c42 (127.5.108.130:44887)
I20260806 01:00:30.910830 6783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41bfb493e52946cabd1828c25774b8d1" candidate_uuid: "ae7f466276b84fa48b7848680b9b592c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d56cefc3b0b414b9e4513c65c152c42"
I20260806 01:00:30.910838 6981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41bfb493e52946cabd1828c25774b8d1" candidate_uuid: "ae7f466276b84fa48b7848680b9b592c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb47395a642945119c57029b3eab8d67"
I20260806 01:00:30.911069 6981 raft_consensus.cc:3037] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:30.911069 6783 raft_consensus.cc:3037] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:30.914309 6981 raft_consensus.cc:2454] T 41bfb493e52946cabd1828c25774b8d1 P cb47395a642945119c57029b3eab8d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae7f466276b84fa48b7848680b9b592c in term 1.
I20260806 01:00:30.914309 6783 raft_consensus.cc:2454] T 41bfb493e52946cabd1828c25774b8d1 P 1d56cefc3b0b414b9e4513c65c152c42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae7f466276b84fa48b7848680b9b592c in term 1.
I20260806 01:00:30.914719 6497 leader_election.cc:302] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d56cefc3b0b414b9e4513c65c152c42, ae7f466276b84fa48b7848680b9b592c; no voters:
I20260806 01:00:30.914996 7086 raft_consensus.cc:2781] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:30.915458 7086 raft_consensus.cc:686] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [term 1 LEADER]: Becoming Leader. State: Replica: ae7f466276b84fa48b7848680b9b592c, State: Running, Role: LEADER
I20260806 01:00:30.915894 7086 consensus_queue.cc:237] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } }
I20260806 01:00:30.921283 6383 catalog_manager.cc:5495] T 41bfb493e52946cabd1828c25774b8d1 P ae7f466276b84fa48b7848680b9b592c reported cstate change: term changed from 0 to 1, leader changed from <none> to ae7f466276b84fa48b7848680b9b592c (127.5.108.129). New cstate: current_term: 1 leader_uuid: "ae7f466276b84fa48b7848680b9b592c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb47395a642945119c57029b3eab8d67" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae7f466276b84fa48b7848680b9b592c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 34211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d56cefc3b0b414b9e4513c65c152c42" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44887 } health_report { overall_health: UNKNOWN } } }
Aug 06 01:00:30 dist-test-slave-x4qh krb5kdc[6317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:00:30.960355 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.945798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.5.108.129:34211)
0806 01:00:30.946006 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:30.946012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:30.946531 (+ 519us) server_negotiation.cc:407] Connection header received
0806 01:00:30.946656 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:30.946661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:30.946728 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:30.946823 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:30.948493 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.949517 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.950703 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:30.951349 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:30.951486 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:30.955309 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:30.955320 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:30.955333 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:30.955405 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:30.957987 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:30.958600 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:30.958606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:30.958608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:30.958751 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:30.959080 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:30.959083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:30.959085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:30.959436 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:30.959700 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:30.959904 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 01:00:30.960073 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 01:00:30 dist-test-slave-x4qh krb5kdc[6317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978030, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:00:31 dist-test-slave-x4qh krb5kdc[6317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978030, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:00:31.016418 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:30.998950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.5.108.129:34211)
0806 01:00:30.999149 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:30.999157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:30.999620 (+ 463us) server_negotiation.cc:407] Connection header received
0806 01:00:30.999834 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:30.999838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:30.999899 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:31.000003 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:31.001303 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.002050 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.003579 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.004257 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.004439 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:31.009315 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:31.009322 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:31.009328 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:31.009372 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:31.013691 (+ 4319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.014325 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.014332 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.014339 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.014480 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.014925 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.014929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.014932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.015319 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:31.015575 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:31.016001 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 01:00:31.016179 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 01:00:31 dist-test-slave-x4qh krb5kdc[6317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:00:31 dist-test-slave-x4qh krb5kdc[6317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:00:31.063864 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:31.051219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.5.108.129:34211)
0806 01:00:31.051465 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:31.051469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:31.051879 (+ 410us) server_negotiation.cc:407] Connection header received
0806 01:00:31.052087 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:31.052105 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:31.052178 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:31.052287 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:31.053475 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.054146 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.055479 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.055859 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.055985 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:31.059465 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:31.059470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:31.059473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:31.059514 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:31.061850 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.062455 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.062459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.062461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.062576 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.062851 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.062853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.062855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.063061 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:31.063222 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:31.063497 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 01:00:31.063641 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 01:00:31.065938 6530 tablet_service.cc:2908] Rejecting scan request for tablet 41bfb493e52946cabd1828c25774b8d1: Uninitialized: clean time has not yet been initialized
I20260806 01:00:31.069661 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6471
I20260806 01:00:31.081460 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:31.070385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.5.108.129:34211)
0806 01:00:31.070534 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:31.070537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:31.070868 (+ 331us) server_negotiation.cc:407] Connection header received
0806 01:00:31.071008 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:31.071011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:31.071062 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:31.071138 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:31.072303 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.072910 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.074034 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.074615 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.074764 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:31.077153 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:31.077158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:31.077161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:31.077187 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:31.079548 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.080072 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.080075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.080077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.080176 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.080454 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.080457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.080458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.080662 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:31.080832 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:31.081076 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 01:00:31.081221 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 01:00:31.082050 6643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:31.169080 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6471
I20260806 01:00:31.194833 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6660
Aug 06 01:00:31 dist-test-slave-x4qh krb5kdc[6317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:00:31.208957 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:31.195748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.5.108.130:44887)
0806 01:00:31.195911 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:31.195915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:31.195932 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:00:31.196086 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:31.196090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:31.196151 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:31.196244 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:31.197338 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.198020 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.199171 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.199651 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.199742 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:31.203710 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:31.203721 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:31.203745 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:31.203775 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:31.206271 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.206827 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.206833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.206835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.206967 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.207308 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.207311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.207313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.207871 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:31.208088 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:31.208399 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 01:00:31.208564 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:00:31.209568 6839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:31.291002 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6660
I20260806 01:00:31.317824 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6853
Aug 06 01:00:31 dist-test-slave-x4qh krb5kdc[6317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:00:31.332989 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:31.318566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.5.108.131:40843)
0806 01:00:31.318736 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:31.318740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:31.318758 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:00:31.318898 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:31.318902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:31.318963 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:31.319053 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:31.320292 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.321112 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.322385 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:31.322818 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:31.322925 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:31.326903 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:31.326916 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:31.326932 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:31.326962 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:31.330193 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.330737 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.330743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.330745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.330892 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:31.331482 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:31.331491 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:31.331495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:31.331938 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:31.332243 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:31.332425 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 01:00:31.332580 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:00:31.333585 7038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:31.415122 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6853
I20260806 01:00:31.441104 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6337
I20260806 01:00:31.442270 6427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:31.548673 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6337
2026-08-06T01:00:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4428 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:00:31 dist-test-slave-x4qh krb5kdc[7122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:00:31 dist-test-slave-x4qh krb5kdc[7122](info): set up 2 sockets
Aug 06 01:00:31 dist-test-slave-x4qh krb5kdc[7122](info): commencing operation
krb5kdc: starting...
W20260806 01:00:32.766932 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:00:32 dist-test-slave-x4qh krb5kdc[7122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:00:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:00:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:00:32.948058 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:45091
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:45091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:33.265734 7141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:33.266012 7141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:33.266112 7141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:33.273177 7141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:00:33.273273 7141 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 01:00:33.273324 7141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:33.273437 7141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:33.273532 7141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:00:33.273686 7141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:00:33.283133 7141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:45091
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:45091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:33.284317 7141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:33.285666 7141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:33.297971 7147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:33.298082 7148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:33.298118 7153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:33.299208 7141 server_base.cc:1034] running on GCE node
I20260806 01:00:33.300056 7141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:33.301420 7141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:33.302693 7141 hybrid_clock.cc:648] HybridClock initialized: now 1785978033302586 us; error 39 us; skew 500 ppm
I20260806 01:00:33.303037 7141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:33 dist-test-slave-x4qh krb5kdc[7122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978033, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:33.306290 7141 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:00:33.307852 7141 webserver.cc:460] Webserver started at http://127.5.108.190:39919/ using document root <none> and password file <none>
I20260806 01:00:33.308542 7141 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:33.308674 7141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:33.309039 7141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:33.312356 7141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "8b23e20e061a4cdfbabcb40fe5c74e42"
format_stamp: "Formatted at 2026-08-06 01:00:33 on dist-test-slave-x4qh"
server_key: "755f23679285a22f2ce62b54df6d7ed6"
server_key_iv: "8560832c414b1e865b7e78c7373c866d"
server_key_version: "encryptionkey@0"
I20260806 01:00:33.313215 7141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "8b23e20e061a4cdfbabcb40fe5c74e42"
format_stamp: "Formatted at 2026-08-06 01:00:33 on dist-test-slave-x4qh"
server_key: "755f23679285a22f2ce62b54df6d7ed6"
server_key_iv: "8560832c414b1e865b7e78c7373c866d"
server_key_version: "encryptionkey@0"
I20260806 01:00:33.318342 7141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:00:33.321827 7161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:33.323099 7141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:00:33.323302 7141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "8b23e20e061a4cdfbabcb40fe5c74e42"
format_stamp: "Formatted at 2026-08-06 01:00:33 on dist-test-slave-x4qh"
server_key: "755f23679285a22f2ce62b54df6d7ed6"
server_key_iv: "8560832c414b1e865b7e78c7373c866d"
server_key_version: "encryptionkey@0"
I20260806 01:00:33.323570 7141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:33.350903 7141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:33.354971 7141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:33.355249 7141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:33.376011 7141 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:45091
I20260806 01:00:33.376042 7251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:45091 every 8 connection(s)
I20260806 01:00:33.377256 7141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:00:33.380651 7252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:33.383355 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 7141
I20260806 01:00:33.383590 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:00:33.384346 5554 external_mini_cluster.cc:1428] Setting key 5f75094db8af880506cc017ef54754fc
I20260806 01:00:33.392609 7252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42: Bootstrap starting.
I20260806 01:00:33.396144 7252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:33.397451 7252 log.cc:824] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:00:33 dist-test-slave-x4qh krb5kdc[7122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:33.400141 7252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42: No bootstrap required, opened a new log
I20260806 01:00:33.406564 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:33.386344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45576 (local address 127.5.108.190:45091)
0806 01:00:33.387084 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:33.387094 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:33.387131 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:00:33.388774 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:33.388798 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:33.389117 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:33.389427 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:33.391108 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.392477 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.394109 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.394763 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.394879 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:33.399707 (+ 4828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:33.399714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:33.399725 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:33.399763 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:33.402750 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:33.403412 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:33.403421 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:33.403429 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:33.403609 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:33.403980 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:33.403992 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:33.403994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:33.404504 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:33.404831 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:33.405246 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 01:00:33.405630 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":345,"threads_started":1}
I20260806 01:00:33.407943 7252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e20e061a4cdfbabcb40fe5c74e42" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45091 } }
I20260806 01:00:33.408275 7252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:33.408375 7252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b23e20e061a4cdfbabcb40fe5c74e42, State: Initialized, Role: FOLLOWER
I20260806 01:00:33.408912 7252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e20e061a4cdfbabcb40fe5c74e42" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45091 } }
I20260806 01:00:33.409147 7252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:00:33.409314 7252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:00:33.409495 7252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:33.411633 7252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e20e061a4cdfbabcb40fe5c74e42" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45091 } }
I20260806 01:00:33.412151 7252 leader_election.cc:302] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b23e20e061a4cdfbabcb40fe5c74e42; no voters:
I20260806 01:00:33.412806 7252 leader_election.cc:288] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:00:33.412909 7261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:33.413616 7261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [term 1 LEADER]: Becoming Leader. State: Replica: 8b23e20e061a4cdfbabcb40fe5c74e42, State: Running, Role: LEADER
I20260806 01:00:33.414165 7261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e20e061a4cdfbabcb40fe5c74e42" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45091 } }
I20260806 01:00:33.414934 7252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:00:33.416939 7263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b23e20e061a4cdfbabcb40fe5c74e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e20e061a4cdfbabcb40fe5c74e42" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45091 } } }
I20260806 01:00:33.417325 7263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:33.419579 7265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b23e20e061a4cdfbabcb40fe5c74e42. Latest consensus state: current_term: 1 leader_uuid: "8b23e20e061a4cdfbabcb40fe5c74e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e20e061a4cdfbabcb40fe5c74e42" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45091 } } }
I20260806 01:00:33.419899 7265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42 [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:33.422763 7273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:00:33.428066 7273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:00:33.436923 7273 catalog_manager.cc:1347] Generated new cluster ID: f4a3887a21884e989af3be5af1b11d33
I20260806 01:00:33.437016 7273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:00:33.451401 7273 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:00:33.453032 7273 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:00:33.463104 7273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42: Generated new TSK 0
I20260806 01:00:33.463989 7273 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:00:33.518357 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:45091
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:33.859331 7291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:33.859617 7291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:33.859767 7291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:33.866685 7291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:33.866812 7291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:33.866966 7291 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:00:33.867069 7291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:00:33.876083 7291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.108.190:45091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:33.877215 7291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:33.878505 7291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:33.891316 7305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:33.891353 7301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:33.891405 7302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:33.892410 7291 server_base.cc:1034] running on GCE node
I20260806 01:00:33.893051 7291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:33.893893 7291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:33.895114 7291 hybrid_clock.cc:648] HybridClock initialized: now 1785978033895034 us; error 46 us; skew 500 ppm
I20260806 01:00:33.895488 7291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:33 dist-test-slave-x4qh krb5kdc[7122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978033, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:33.904266 7291 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:00:33.905725 7291 webserver.cc:460] Webserver started at http://127.5.108.129:34807/ using document root <none> and password file <none>
I20260806 01:00:33.906394 7291 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:33.906517 7291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:33.906848 7291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:33.909857 7291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "5609ef976d46425f9aa8709c1fd0809d"
format_stamp: "Formatted at 2026-08-06 01:00:33 on dist-test-slave-x4qh"
server_key: "24964e8dceb0864af2a206623afd36a4"
server_key_iv: "421f8cc3ce7569d5da3942dfce72625e"
server_key_version: "encryptionkey@0"
I20260806 01:00:33.910629 7291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "5609ef976d46425f9aa8709c1fd0809d"
format_stamp: "Formatted at 2026-08-06 01:00:33 on dist-test-slave-x4qh"
server_key: "24964e8dceb0864af2a206623afd36a4"
server_key_iv: "421f8cc3ce7569d5da3942dfce72625e"
server_key_version: "encryptionkey@0"
I20260806 01:00:33.915804 7291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 01:00:33.919111 7316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:33.920491 7291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:00:33.920643 7291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "5609ef976d46425f9aa8709c1fd0809d"
format_stamp: "Formatted at 2026-08-06 01:00:33 on dist-test-slave-x4qh"
server_key: "24964e8dceb0864af2a206623afd36a4"
server_key_iv: "421f8cc3ce7569d5da3942dfce72625e"
server_key_version: "encryptionkey@0"
I20260806 01:00:33.920857 7291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:33.939926 7291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:33.943760 7291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:33.944023 7291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:33.946597 7291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:33.946709 7291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:33.946856 7291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:33.946925 7291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:00:33 dist-test-slave-x4qh krb5kdc[7122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978033, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:33.970939 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:33.947367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:54403 (local address 127.5.108.190:45091)
0806 01:00:33.947844 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:33.947848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:33.948985 (+ 1137us) server_negotiation.cc:407] Connection header received
0806 01:00:33.950933 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:33.950939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:33.951022 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:33.951171 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:33.953433 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.954516 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.957305 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.957886 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.958025 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:33.963449 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:33.963459 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:33.963466 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:33.963513 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:33.966858 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:33.968114 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:33.968121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:33.968125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:33.968273 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:33.968954 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:33.968958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:33.968960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:33.969297 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:33.969466 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:33.970508 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0806 01:00:33.970684 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":248,"threads_started":1}
I20260806 01:00:33.973055 7329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:33.948027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45091 (local address 127.5.108.129:54403)
0806 01:00:33.948829 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:33.948874 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:33.950668 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:33.951472 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:33.951480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:33.952221 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:33.953153 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:33.953186 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.954664 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.954671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:33.957154 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:33.957166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.958171 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.958180 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:33.958385 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:33.959229 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:33.959274 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:33.963132 (+ 3858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:33.967089 (+ 3957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:33.967098 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:33.967122 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:33.967943 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:33.968390 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:33.968396 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:33.968399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:33.968785 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:33.969464 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:33.969476 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:33.970030 (+ 554us) client_negotiation.cc:769] Sending connection context
0806 01:00:33.970424 (+ 394us) client_negotiation.cc:240] Negotiation successful
0806 01:00:33.971097 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":363,"threads_started":1}
I20260806 01:00:33.992909 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:33.979001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:59531 (local address 127.5.108.190:45091)
0806 01:00:33.979211 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:33.979219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:33.979920 (+ 701us) server_negotiation.cc:407] Connection header received
0806 01:00:33.980189 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:33.980205 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:33.980297 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:33.980404 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:33.981868 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.982724 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.984474 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.984886 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.985019 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:33.986903 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:33.986908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:33.986911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:33.986940 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:33.989939 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:33.990824 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:33.990832 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:33.990837 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:33.990995 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:33.991487 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:33.991494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:33.991498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:33.991772 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:33.991927 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:33.992358 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 01:00:33.992570 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:00:33.993014 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:33.979004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45091 (local address 127.5.108.129:59531)
0806 01:00:33.979788 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:33.979812 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:33.979980 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:33.980467 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:33.980474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:33.981217 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:33.981710 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:33.981727 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.982863 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.982868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:33.984299 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:33.984315 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:33.985075 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:33.985083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:33.985241 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:33.985953 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:33.985971 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:33.986697 (+ 726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:33.990075 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:33.990080 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:33.990084 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:33.990609 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:33.991140 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:33.991150 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:33.991178 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:33.991321 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:33.991892 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:33.991897 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:33.992080 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 01:00:33.992236 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 01:00:33.992566 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":398,"threads_started":1}
I20260806 01:00:34.002406 7291 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:36103
I20260806 01:00:34.002432 7495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:36103 every 8 connection(s)
I20260806 01:00:34.003645 7291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:00:34.005723 7497 heartbeater.cc:347] Connected to a master server at 127.5.108.190:45091
I20260806 01:00:34.005956 7497 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:34.006609 7497 heartbeater.cc:511] Master 127.5.108.190:45091 requested a full tablet report, sending...
I20260806 01:00:34.008486 7192 ts_manager.cc:194] Registered new tserver with Master: 5609ef976d46425f9aa8709c1fd0809d (127.5.108.129:36103)
I20260806 01:00:34.009822 7192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:54403
I20260806 01:00:34.011778 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 7291
I20260806 01:00:34.011998 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:00:34.012687 5554 external_mini_cluster.cc:1428] Setting key 0ebc64a7e49aac60d8882c4810d71c8e
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:00:34.054741 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:45091
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:34.378580 7502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:34.378930 7502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:34.379091 7502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:34.390555 7502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:34.390686 7502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:34.390825 7502 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:00:34.390972 7502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:00:34.403271 7502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.108.190:45091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:34.404481 7502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:34.405759 7502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:34.418509 7512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:34.418473 7515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:34.419306 7502 server_base.cc:1034] running on GCE node
W20260806 01:00:34.419315 7511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:34.420135 7502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:34.421018 7502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:34.422263 7502 hybrid_clock.cc:648] HybridClock initialized: now 1785978034422180 us; error 53 us; skew 500 ppm
I20260806 01:00:34.422557 7502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:34 dist-test-slave-x4qh krb5kdc[7122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978034, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:34.425737 7502 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:00:34.427145 7502 webserver.cc:460] Webserver started at http://127.5.108.130:44899/ using document root <none> and password file <none>
I20260806 01:00:34.427920 7502 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:34.428056 7502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:34.428387 7502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:34.431034 7502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "62a4f677190f480ba7fed2d6451f7ce0"
format_stamp: "Formatted at 2026-08-06 01:00:34 on dist-test-slave-x4qh"
server_key: "376c4c148ecf98ab6eb13ead5c90fd18"
server_key_iv: "d14aac26d50076cf43781eeac7681a74"
server_key_version: "encryptionkey@0"
I20260806 01:00:34.431871 7502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "62a4f677190f480ba7fed2d6451f7ce0"
format_stamp: "Formatted at 2026-08-06 01:00:34 on dist-test-slave-x4qh"
server_key: "376c4c148ecf98ab6eb13ead5c90fd18"
server_key_iv: "d14aac26d50076cf43781eeac7681a74"
server_key_version: "encryptionkey@0"
I20260806 01:00:34.436736 7502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 01:00:34.440064 7525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:34.441253 7502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:00:34.441433 7502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "62a4f677190f480ba7fed2d6451f7ce0"
format_stamp: "Formatted at 2026-08-06 01:00:34 on dist-test-slave-x4qh"
server_key: "376c4c148ecf98ab6eb13ead5c90fd18"
server_key_iv: "d14aac26d50076cf43781eeac7681a74"
server_key_version: "encryptionkey@0"
I20260806 01:00:34.441671 7502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:34.467934 7502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:34.471832 7502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:34.472114 7502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:34.474774 7502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:34.474881 7502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:34.475028 7502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:34.475100 7502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:00:34 dist-test-slave-x4qh krb5kdc[7122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978034, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:34.499051 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:34.475624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:44641 (local address 127.5.108.190:45091)
0806 01:00:34.475844 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:34.475850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:34.477354 (+ 1504us) server_negotiation.cc:407] Connection header received
0806 01:00:34.479818 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:34.479829 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:34.479944 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:34.480071 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:34.482392 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.483390 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.485828 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.486615 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.486749 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:34.491013 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:34.491019 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:34.491024 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:34.491064 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:34.495109 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:34.496067 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:34.496071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:34.496072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:34.496203 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:34.496886 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:34.496894 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:34.496897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:34.497379 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:34.497561 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:34.498443 (+ 882us) server_negotiation.cc:299] Negotiation successful
0806 01:00:34.498700 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:00:34.500406 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:34.476236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45091 (local address 127.5.108.130:44641)
0806 01:00:34.477142 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:34.477196 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:34.479292 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:34.480359 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:34.480375 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:34.481178 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:34.482140 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:34.482170 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.483578 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.483581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:34.485676 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:34.485688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.486843 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.486852 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:34.487089 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:34.487853 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:34.487890 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:34.490806 (+ 2916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:34.495262 (+ 4456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:34.495268 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:34.495290 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:34.495908 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:34.496366 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:34.496370 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:34.496377 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:34.496698 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:34.497554 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:34.497563 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:34.498050 (+ 487us) client_negotiation.cc:769] Sending connection context
0806 01:00:34.498323 (+ 273us) client_negotiation.cc:240] Negotiation successful
0806 01:00:34.498847 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":425,"threads_started":1}
I20260806 01:00:34.524801 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:34.507261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45091 (local address 127.5.108.130:41975)
0806 01:00:34.508329 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:34.508352 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:34.508512 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:34.508834 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:34.508841 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:34.509558 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:34.510023 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:34.510039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.510997 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.511004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:34.513375 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:34.513396 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.514452 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.514456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:34.514683 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:34.515489 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:34.515510 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:34.516553 (+ 1043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:34.520930 (+ 4377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:34.520938 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:34.520943 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:34.521466 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:34.522043 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:34.522046 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:34.522049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:34.522257 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:34.523289 (+ 1032us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:34.523297 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:34.523596 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 01:00:34.523813 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 01:00:34.524221 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"thread_start_us":721,"threads_started":1}
I20260806 01:00:34.525879 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:34.507254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:41975 (local address 127.5.108.190:45091)
0806 01:00:34.507487 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:34.507494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:34.508445 (+ 951us) server_negotiation.cc:407] Connection header received
0806 01:00:34.508630 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:34.508633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:34.508696 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:34.508770 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:34.510178 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.510858 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.513545 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.514067 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.514205 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:34.516784 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:34.516793 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:34.516800 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:34.516846 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:34.520728 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:34.521665 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:34.521672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:34.521675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:34.521845 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:34.522427 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:34.522436 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:34.522453 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:34.523021 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:34.523202 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:34.525157 (+ 1955us) server_negotiation.cc:299] Negotiation successful
0806 01:00:34.525469 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:00:34.526443 7502 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:44833
I20260806 01:00:34.526537 7703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:44833 every 8 connection(s)
I20260806 01:00:34.528051 7502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:00:34.530232 7704 heartbeater.cc:347] Connected to a master server at 127.5.108.190:45091
I20260806 01:00:34.530476 7704 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:34.531059 7704 heartbeater.cc:511] Master 127.5.108.190:45091 requested a full tablet report, sending...
I20260806 01:00:34.532476 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 7502
I20260806 01:00:34.532527 7192 ts_manager.cc:194] Registered new tserver with Master: 62a4f677190f480ba7fed2d6451f7ce0 (127.5.108.130:44833)
I20260806 01:00:34.532711 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:00:34.533255 5554 external_mini_cluster.cc:1428] Setting key 1d46663ea4e5b281449b148776bad732
I20260806 01:00:34.533576 7192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:44641
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:00:34.576828 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:45091
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:34.899195 7709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:34.899503 7709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:34.899610 7709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:34.906646 7709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:34.906742 7709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:34.906824 7709 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:00:34.906901 7709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:00:34.916354 7709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.108.190:45091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:34.917510 7709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:34.918802 7709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:34.931975 7718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:34.931927 7722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:34.932013 7719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:34.933076 7709 server_base.cc:1034] running on GCE node
I20260806 01:00:34.933583 7709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:34.934301 7709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:34.935487 7709 hybrid_clock.cc:648] HybridClock initialized: now 1785978034935424 us; error 23 us; skew 500 ppm
I20260806 01:00:34.935783 7709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:34 dist-test-slave-x4qh krb5kdc[7122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978034, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:34.938846 7709 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:00:34.940281 7709 webserver.cc:460] Webserver started at http://127.5.108.131:39971/ using document root <none> and password file <none>
I20260806 01:00:34.940871 7709 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:34.940953 7709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:34.941210 7709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:34.944123 7709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "32db6efb919b4fc8aadf9d3060c3eb13"
format_stamp: "Formatted at 2026-08-06 01:00:34 on dist-test-slave-x4qh"
server_key: "8207be0b32a24e929e165fbe875c5c58"
server_key_iv: "e69822e96e7154ef7cf10583a94dee4e"
server_key_version: "encryptionkey@0"
I20260806 01:00:34.944927 7709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "32db6efb919b4fc8aadf9d3060c3eb13"
format_stamp: "Formatted at 2026-08-06 01:00:34 on dist-test-slave-x4qh"
server_key: "8207be0b32a24e929e165fbe875c5c58"
server_key_iv: "e69822e96e7154ef7cf10583a94dee4e"
server_key_version: "encryptionkey@0"
I20260806 01:00:34.949566 7709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:00:34.952867 7733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:34.954037 7709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:00:34.954203 7709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "32db6efb919b4fc8aadf9d3060c3eb13"
format_stamp: "Formatted at 2026-08-06 01:00:34 on dist-test-slave-x4qh"
server_key: "8207be0b32a24e929e165fbe875c5c58"
server_key_iv: "e69822e96e7154ef7cf10583a94dee4e"
server_key_version: "encryptionkey@0"
I20260806 01:00:34.954417 7709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:34.981652 7709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:34.985553 7709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:34.985826 7709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:34.988483 7709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:34.988590 7709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:34.988783 7709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:34.988862 7709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:00:34 dist-test-slave-x4qh krb5kdc[7122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978034, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:35.011493 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:34.988960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:42933 (local address 127.5.108.190:45091)
0806 01:00:34.989168 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:34.989177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:34.990427 (+ 1250us) server_negotiation.cc:407] Connection header received
0806 01:00:34.992503 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:34.992509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:34.992596 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:34.992698 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:34.994761 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.996109 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.998800 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.999259 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.999425 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.003679 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:35.003685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:35.003688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:35.003760 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:35.007467 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.008512 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.008520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.008524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.008700 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.009284 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.009290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.009295 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.009705 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:35.009874 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.010866 (+ 992us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.011090 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 01:00:35.012439 7497 heartbeater.cc:503] Master 127.5.108.190:45091 was elected leader, sending a full tablet report...
I20260806 01:00:35.012528 7751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:34.989509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45091 (local address 127.5.108.131:42933)
0806 01:00:34.990260 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:34.990306 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:34.992205 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:34.992929 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:34.992958 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:34.993661 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:34.994495 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:34.994529 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.996320 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.996331 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:34.998570 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:34.998585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:34.999565 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:34.999573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:34.999781 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.000543 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:35.000581 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:35.003316 (+ 2735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:35.007651 (+ 4335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:35.007661 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:35.007676 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:35.008341 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:35.008842 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:35.008849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:35.008855 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:35.009132 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:35.009966 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:35.009974 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:35.010272 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.010647 (+ 375us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.011104 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":316,"threads_started":1}
I20260806 01:00:35.035231 7709 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:44181
I20260806 01:00:35.035285 7905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:44181 every 8 connection(s)
I20260806 01:00:35.035914 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.018575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45091 (local address 127.5.108.131:34865)
0806 01:00:35.019198 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.019222 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.019423 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.019848 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.019859 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.020581 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:35.021082 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.021101 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.022698 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.022705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.024808 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.024825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.025984 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.025993 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.026127 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.026789 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:35.026805 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:35.028155 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:35.032609 (+ 4454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:35.032616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:35.032622 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:35.033298 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:35.033717 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:35.033720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:35.033722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:35.033895 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:35.034703 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:35.034712 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:35.034993 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.035176 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.035564 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":329,"threads_started":1}
I20260806 01:00:35.035940 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.018485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:34865 (local address 127.5.108.190:45091)
0806 01:00:35.018648 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.018654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.019344 (+ 690us) server_negotiation.cc:407] Connection header received
0806 01:00:35.019589 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.019598 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.019697 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.019796 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:35.021284 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.022545 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.024969 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.025748 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.025884 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.028382 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:35.028388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:35.028393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:35.028431 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:35.032445 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.033433 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.033439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.033442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.033596 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.034105 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.034112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.034118 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.034517 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:35.034715 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.035328 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.035581 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 01:00:35.036831 7709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:00:35.039686 7906 heartbeater.cc:347] Connected to a master server at 127.5.108.190:45091
I20260806 01:00:35.040424 7906 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:35.041011 7906 heartbeater.cc:511] Master 127.5.108.190:45091 requested a full tablet report, sending...
I20260806 01:00:35.042250 7192 ts_manager.cc:194] Registered new tserver with Master: 32db6efb919b4fc8aadf9d3060c3eb13 (127.5.108.131:44181)
I20260806 01:00:35.043042 7192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:42933
I20260806 01:00:35.044463 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 7709
I20260806 01:00:35.044664 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:00:35.045141 5554 external_mini_cluster.cc:1428] Setting key a82d9421188864b8b43c7594ad767672
I20260806 01:00:35.047987 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:00:35 dist-test-slave-x4qh krb5kdc[7122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978035, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:00:35 dist-test-slave-x4qh krb5kdc[7122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978035, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:35.097725 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.083951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.5.108.190:45091)
0806 01:00:35.084232 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.084237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.084605 (+ 368us) server_negotiation.cc:407] Connection header received
0806 01:00:35.084783 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.084788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.084854 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.084941 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:35.086247 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.086979 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.088453 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.088882 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.088973 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.092621 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:35.092626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:35.092629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:35.092659 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:35.095348 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.096034 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.096038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.096040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.096158 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.096519 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.096522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.096524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.096789 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:35.096966 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.097272 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.097444 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260806 01:00:35.101905 7192 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:00:35.108871 7192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:00:35.143963 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.122121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38910 (local address 127.5.108.130:44833)
0806 01:00:35.134789 (+ 12668us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.134801 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.135889 (+ 1088us) server_negotiation.cc:407] Connection header received
0806 01:00:35.135989 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.135998 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.136219 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.136398 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:35.137677 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.139122 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.141996 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.142860 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.143016 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.143219 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.143512 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.143678 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12496,"thread_start_us":12378,"threads_started":1}
I20260806 01:00:35.144472 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.121459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44833 (local address 127.0.0.1:38910)
0806 01:00:35.135623 (+ 14164us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.135639 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.135761 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.136599 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.136606 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.136629 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:35.137483 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.137501 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.139272 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.139277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.141854 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.141864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.143048 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.143054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.143241 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.143302 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.143468 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.143654 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14033,"mutex_wait_us":1,"spinlock_wait_cycles":13312,"thread_start_us":9083,"threads_started":1}
I20260806 01:00:35.144913 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.120963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:36103 (local address 127.0.0.1:48376)
0806 01:00:35.133106 (+ 12143us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.133126 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.133351 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.136739 (+ 3388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.136744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.136788 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:35.137523 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.137536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.138841 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.138849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.141575 (+ 2726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.141589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.143653 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.143665 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.143847 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.144286 (+ 439us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.144383 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.144521 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11981,"mutex_wait_us":398,"spinlock_wait_cycles":2304,"thread_start_us":12477,"threads_started":1}
I20260806 01:00:35.145072 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.120968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54466 (local address 127.5.108.131:44181)
0806 01:00:35.135363 (+ 14395us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.135391 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.135427 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:00:35.135528 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.135536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.135777 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.135954 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:35.136958 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.138739 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.141608 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.143082 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.143223 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.143480 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.144559 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.144733 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14202,"thread_start_us":14051,"threads_started":1}
I20260806 01:00:35.145606 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.121537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.5.108.129:36103)
0806 01:00:35.136008 (+ 14471us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.136014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.136034 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:00:35.136104 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.136108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.136263 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.136459 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:35.137701 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.138694 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.141733 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.143161 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.143281 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.143493 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.145196 (+ 1703us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.145359 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14328,"thread_start_us":14197,"threads_started":1}
I20260806 01:00:35.146273 7930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.120639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:44181 (local address 127.0.0.1:54466)
0806 01:00:35.131054 (+ 10415us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.131103 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.131324 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.136281 (+ 4957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.136288 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.136347 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:35.136735 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.136752 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.138896 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.138905 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.141437 (+ 2532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.141450 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.143685 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.143691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.143845 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.144094 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.144179 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.144327 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10100,"mutex_wait_us":94,"thread_start_us":14693,"threads_started":1}
I20260806 01:00:35.148375 7602 tablet_service.cc:1467] Processing CreateTablet for tablet babf0fbbd2cc40d39cebc7337ae11a6f (DEFAULT_TABLE table=test-table [id=795906074d934ce29e7ccba2ef9f0e76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:35.149248 7391 tablet_service.cc:1467] Processing CreateTablet for tablet babf0fbbd2cc40d39cebc7337ae11a6f (DEFAULT_TABLE table=test-table [id=795906074d934ce29e7ccba2ef9f0e76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:35.149796 7602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet babf0fbbd2cc40d39cebc7337ae11a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:35.149740 7812 tablet_service.cc:1467] Processing CreateTablet for tablet babf0fbbd2cc40d39cebc7337ae11a6f (DEFAULT_TABLE table=test-table [id=795906074d934ce29e7ccba2ef9f0e76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:35.150606 7391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet babf0fbbd2cc40d39cebc7337ae11a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:35.151083 7812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet babf0fbbd2cc40d39cebc7337ae11a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:35.160709 7936 tablet_bootstrap.cc:492] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: Bootstrap starting.
I20260806 01:00:35.164130 7936 tablet_bootstrap.cc:654] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:35.165041 7937 tablet_bootstrap.cc:492] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: Bootstrap starting.
I20260806 01:00:35.165508 7936 log.cc:824] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:35.169191 7937 tablet_bootstrap.cc:654] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:35.171025 7939 tablet_bootstrap.cc:492] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: Bootstrap starting.
I20260806 01:00:35.171962 7937 log.cc:824] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:35.174523 7939 tablet_bootstrap.cc:654] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:35.174896 7937 tablet_bootstrap.cc:492] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: No bootstrap required, opened a new log
I20260806 01:00:35.175203 7937 ts_tablet_manager.cc:1397] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 01:00:35.175303 7936 tablet_bootstrap.cc:492] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: No bootstrap required, opened a new log
I20260806 01:00:35.175689 7936 ts_tablet_manager.cc:1397] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20260806 01:00:35.175876 7939 log.cc:824] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:35.178777 7939 tablet_bootstrap.cc:492] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: No bootstrap required, opened a new log
I20260806 01:00:35.179021 7939 ts_tablet_manager.cc:1397] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 01:00:35.183163 7937 raft_consensus.cc:348] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } }
I20260806 01:00:35.183588 7937 raft_consensus.cc:374] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:35.183720 7937 raft_consensus.cc:729] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5609ef976d46425f9aa8709c1fd0809d, State: Initialized, Role: FOLLOWER
I20260806 01:00:35.184314 7937 consensus_queue.cc:260] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } }
I20260806 01:00:35.186059 7937 ts_tablet_manager.cc:1428] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260806 01:00:35.186076 7936 raft_consensus.cc:348] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } }
I20260806 01:00:35.186483 7936 raft_consensus.cc:374] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:35.186618 7936 raft_consensus.cc:729] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a4f677190f480ba7fed2d6451f7ce0, State: Initialized, Role: FOLLOWER
I20260806 01:00:35.187160 7936 consensus_queue.cc:260] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } }
I20260806 01:00:35.188279 7704 heartbeater.cc:503] Master 127.5.108.190:45091 was elected leader, sending a full tablet report...
I20260806 01:00:35.188824 7936 ts_tablet_manager.cc:1428] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260806 01:00:35.189478 7939 raft_consensus.cc:348] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } }
I20260806 01:00:35.189894 7939 raft_consensus.cc:374] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:35.190001 7939 raft_consensus.cc:729] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32db6efb919b4fc8aadf9d3060c3eb13, State: Initialized, Role: FOLLOWER
I20260806 01:00:35.190523 7939 consensus_queue.cc:260] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } }
I20260806 01:00:35.191861 7906 heartbeater.cc:503] Master 127.5.108.190:45091 was elected leader, sending a full tablet report...
I20260806 01:00:35.192523 7939 ts_tablet_manager.cc:1428] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: Time spent starting tablet: real 0.013s user 0.008s sys 0.006s
I20260806 01:00:35.233424 7944 raft_consensus.cc:482] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:35.233659 7944 raft_consensus.cc:504] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } }
I20260806 01:00:35.235105 7944 leader_election.cc:288] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32db6efb919b4fc8aadf9d3060c3eb13 (127.5.108.131:44181), 5609ef976d46425f9aa8709c1fd0809d (127.5.108.129:36103)
I20260806 01:00:35.244203 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.235575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:45323 (local address 127.5.108.129:36103)
0806 01:00:35.235797 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.235805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.236613 (+ 808us) server_negotiation.cc:407] Connection header received
0806 01:00:35.236798 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.236801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.236899 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.237003 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:35.237719 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.238882 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.241787 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.243022 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.243182 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.243255 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.243802 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.243942 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:00:35.244318 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.235629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:36103 (local address 127.5.108.130:45323)
0806 01:00:35.236465 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.236488 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.236658 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.237035 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.237038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.237057 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:35.237544 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.237562 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.239060 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.239065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.241621 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.241630 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.243434 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.243444 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.243644 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.243681 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.243766 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.243881 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":159,"spinlock_wait_cycles":2432,"thread_start_us":459,"threads_started":1}
I20260806 01:00:35.244306 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.235498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:53267 (local address 127.5.108.131:44181)
0806 01:00:35.235712 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.235718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.236879 (+ 1161us) server_negotiation.cc:407] Connection header received
0806 01:00:35.236991 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.236996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.237064 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.237150 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:35.237852 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.238912 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.241779 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.243022 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.243194 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.243255 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.243927 (+ 672us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.244064 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:00:35.244756 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.235720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:44181 (local address 127.5.108.130:53267)
0806 01:00:35.236766 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.236783 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.236891 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.237183 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.237191 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.237228 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:35.237671 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.237682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.239062 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.239065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.241613 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.241630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.243512 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.243521 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.243750 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.243784 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.243879 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.243989 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"mutex_wait_us":120,"thread_start_us":706,"threads_started":1}
I20260806 01:00:35.245455 7423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "babf0fbbd2cc40d39cebc7337ae11a6f" candidate_uuid: "62a4f677190f480ba7fed2d6451f7ce0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5609ef976d46425f9aa8709c1fd0809d" is_pre_election: true
I20260806 01:00:35.245811 7841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "babf0fbbd2cc40d39cebc7337ae11a6f" candidate_uuid: "62a4f677190f480ba7fed2d6451f7ce0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" is_pre_election: true
I20260806 01:00:35.245965 7423 raft_consensus.cc:2454] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a4f677190f480ba7fed2d6451f7ce0 in term 0.
I20260806 01:00:35.246374 7841 raft_consensus.cc:2454] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a4f677190f480ba7fed2d6451f7ce0 in term 0.
I20260806 01:00:35.246614 7531 leader_election.cc:302] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5609ef976d46425f9aa8709c1fd0809d, 62a4f677190f480ba7fed2d6451f7ce0; no voters:
I20260806 01:00:35.247016 7944 raft_consensus.cc:2781] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:00:35.247182 7944 raft_consensus.cc:482] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:00:35.247294 7944 raft_consensus.cc:3037] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:35.249444 7944 raft_consensus.cc:504] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } }
I20260806 01:00:35.250041 7944 leader_election.cc:288] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [CANDIDATE]: Term 1 election: Requested vote from peers 32db6efb919b4fc8aadf9d3060c3eb13 (127.5.108.131:44181), 5609ef976d46425f9aa8709c1fd0809d (127.5.108.129:36103)
I20260806 01:00:35.250353 7841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "babf0fbbd2cc40d39cebc7337ae11a6f" candidate_uuid: "62a4f677190f480ba7fed2d6451f7ce0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32db6efb919b4fc8aadf9d3060c3eb13"
I20260806 01:00:35.250536 7841 raft_consensus.cc:3037] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:35.250509 7423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "babf0fbbd2cc40d39cebc7337ae11a6f" candidate_uuid: "62a4f677190f480ba7fed2d6451f7ce0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5609ef976d46425f9aa8709c1fd0809d"
I20260806 01:00:35.250715 7423 raft_consensus.cc:3037] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:35.253134 7841 raft_consensus.cc:2454] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a4f677190f480ba7fed2d6451f7ce0 in term 1.
I20260806 01:00:35.253208 7423 raft_consensus.cc:2454] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a4f677190f480ba7fed2d6451f7ce0 in term 1.
I20260806 01:00:35.253574 7531 leader_election.cc:302] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5609ef976d46425f9aa8709c1fd0809d, 62a4f677190f480ba7fed2d6451f7ce0; no voters:
I20260806 01:00:35.253854 7944 raft_consensus.cc:2781] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:35.254290 7944 raft_consensus.cc:686] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 LEADER]: Becoming Leader. State: Replica: 62a4f677190f480ba7fed2d6451f7ce0, State: Running, Role: LEADER
I20260806 01:00:35.254722 7944 consensus_queue.cc:237] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } }
W20260806 01:00:35.256003 7498 tablet.cc:2369] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:00:35.258832 7188 catalog_manager.cc:5495] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a4f677190f480ba7fed2d6451f7ce0 (127.5.108.130). New cstate: current_term: 1 leader_uuid: "62a4f677190f480ba7fed2d6451f7ce0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } health_report { overall_health: HEALTHY } } }
W20260806 01:00:35.280954 7705 tablet.cc:2369] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:00:35.290167 7907 tablet.cc:2369] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:00:35.305555 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.300080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.5.108.130:44833)
0806 01:00:35.300289 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.300293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.300313 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:00:35.300627 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.300630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.300714 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.300846 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:35.301172 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.301994 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.303348 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.303827 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.303921 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.304379 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:35.304926 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:35.305039 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.305166 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.305252 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:00:35.317507 7841 raft_consensus.cc:1261] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Refusing update from remote peer 62a4f677190f480ba7fed2d6451f7ce0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:35.318701 7950 consensus_queue.cc:1035] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:35.319826 7423 raft_consensus.cc:1261] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Refusing update from remote peer 62a4f677190f480ba7fed2d6451f7ce0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:35.320593 7950 consensus_queue.cc:1035] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:35.334622 7954 mvcc.cc:205] Tried to move back new op lower bound from 7315366032635580416 to 7315366032407134208. Current Snapshot: MvccSnapshot[applied={T|T < 7315366032635580416}]
I20260806 01:00:35.352250 7192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:35.352591 7192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:35.356700 7192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42: Sending DeleteTablet for 3 replicas of tablet babf0fbbd2cc40d39cebc7337ae11a6f
I20260806 01:00:35.358202 7812 tablet_service.cc:1514] Processing DeleteTablet for tablet babf0fbbd2cc40d39cebc7337ae11a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:35 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:54466
I20260806 01:00:35.358170 7391 tablet_service.cc:1514] Processing DeleteTablet for tablet babf0fbbd2cc40d39cebc7337ae11a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:35 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:48376
I20260806 01:00:35.359354 7963 tablet_replica.cc:330] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: stopping tablet replica
I20260806 01:00:35.359769 7602 tablet_service.cc:1514] Processing DeleteTablet for tablet babf0fbbd2cc40d39cebc7337ae11a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:35 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:38910
I20260806 01:00:35.360121 7963 raft_consensus.cc:2229] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:35.360746 7963 raft_consensus.cc:2258] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:35.363426 7963 ts_tablet_manager.cc:1905] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:35.363364 7188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:00:35.364197 7188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:00:35.371840 7391 tablet_service.cc:1467] Processing CreateTablet for tablet fe9b3e5f654f4da9b6b08db96ff4d30b (DEFAULT_TABLE table=test-table [id=6f19cb6801b94e9c8559b3b60e1e6ab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:35.372359 7391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9b3e5f654f4da9b6b08db96ff4d30b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:35.373028 7963 ts_tablet_manager.cc:1918] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:00:35.373400 7963 log.cc:1196] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/babf0fbbd2cc40d39cebc7337ae11a6f
I20260806 01:00:35.373514 7811 tablet_service.cc:1467] Processing CreateTablet for tablet fe9b3e5f654f4da9b6b08db96ff4d30b (DEFAULT_TABLE table=test-table [id=6f19cb6801b94e9c8559b3b60e1e6ab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:35.374095 7811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9b3e5f654f4da9b6b08db96ff4d30b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:35.374459 7963 ts_tablet_manager.cc:1939] T babf0fbbd2cc40d39cebc7337ae11a6f P 5609ef976d46425f9aa8709c1fd0809d: Deleting consensus metadata
I20260806 01:00:35.375149 7965 tablet_replica.cc:330] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: stopping tablet replica
I20260806 01:00:35.375749 7965 raft_consensus.cc:2229] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:00:35.376667 7965 raft_consensus.cc:2258] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:35.376668 7168 catalog_manager.cc:4841] TS 5609ef976d46425f9aa8709c1fd0809d (127.5.108.129:36103): tablet babf0fbbd2cc40d39cebc7337ae11a6f (table test-table [id=795906074d934ce29e7ccba2ef9f0e76]) successfully deleted
I20260806 01:00:35.379197 7965 ts_tablet_manager.cc:1905] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:35.379424 7937 tablet_bootstrap.cc:492] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: Bootstrap starting.
I20260806 01:00:35.381021 7937 tablet_bootstrap.cc:654] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:35.384271 7602 tablet_service.cc:1467] Processing CreateTablet for tablet fe9b3e5f654f4da9b6b08db96ff4d30b (DEFAULT_TABLE table=test-table [id=6f19cb6801b94e9c8559b3b60e1e6ab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:35.384608 7964 tablet_replica.cc:330] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: stopping tablet replica
I20260806 01:00:35.384879 7602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9b3e5f654f4da9b6b08db96ff4d30b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:35.385067 7964 raft_consensus.cc:2229] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:35.385551 7964 raft_consensus.cc:2258] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:35.387637 7964 ts_tablet_manager.cc:1905] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:35.388163 7965 ts_tablet_manager.cc:1918] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:00:35.388393 7965 log.cc:1196] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/babf0fbbd2cc40d39cebc7337ae11a6f
I20260806 01:00:35.389043 7965 ts_tablet_manager.cc:1939] T babf0fbbd2cc40d39cebc7337ae11a6f P 62a4f677190f480ba7fed2d6451f7ce0: Deleting consensus metadata
I20260806 01:00:35.390684 7164 catalog_manager.cc:4841] TS 62a4f677190f480ba7fed2d6451f7ce0 (127.5.108.130:44833): tablet babf0fbbd2cc40d39cebc7337ae11a6f (table test-table [id=795906074d934ce29e7ccba2ef9f0e76]) successfully deleted
I20260806 01:00:35.392864 7937 tablet_bootstrap.cc:492] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: No bootstrap required, opened a new log
I20260806 01:00:35.393076 7937 ts_tablet_manager.cc:1397] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20260806 01:00:35.393929 7937 raft_consensus.cc:348] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.394282 7937 raft_consensus.cc:374] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:35.394440 7937 raft_consensus.cc:729] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5609ef976d46425f9aa8709c1fd0809d, State: Initialized, Role: FOLLOWER
I20260806 01:00:35.394776 7964 ts_tablet_manager.cc:1918] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:00:35.394981 7964 log.cc:1196] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/babf0fbbd2cc40d39cebc7337ae11a6f
I20260806 01:00:35.394932 7937 consensus_queue.cc:260] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.395865 7937 ts_tablet_manager.cc:1428] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 01:00:35.396283 7964 ts_tablet_manager.cc:1939] T babf0fbbd2cc40d39cebc7337ae11a6f P 32db6efb919b4fc8aadf9d3060c3eb13: Deleting consensus metadata
I20260806 01:00:35.396461 7939 tablet_bootstrap.cc:492] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: Bootstrap starting.
I20260806 01:00:35.398053 7936 tablet_bootstrap.cc:492] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: Bootstrap starting.
I20260806 01:00:35.398272 7939 tablet_bootstrap.cc:654] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:35.399907 7936 tablet_bootstrap.cc:654] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:35.400492 7163 catalog_manager.cc:4841] TS 32db6efb919b4fc8aadf9d3060c3eb13 (127.5.108.131:44181): tablet babf0fbbd2cc40d39cebc7337ae11a6f (table test-table [id=795906074d934ce29e7ccba2ef9f0e76]) successfully deleted
I20260806 01:00:35.405030 7939 tablet_bootstrap.cc:492] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: No bootstrap required, opened a new log
I20260806 01:00:35.405196 7939 ts_tablet_manager.cc:1397] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 01:00:35.406020 7939 raft_consensus.cc:348] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.406267 7939 raft_consensus.cc:374] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:35.406371 7939 raft_consensus.cc:729] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32db6efb919b4fc8aadf9d3060c3eb13, State: Initialized, Role: FOLLOWER
I20260806 01:00:35.406641 7939 consensus_queue.cc:260] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.407334 7939 ts_tablet_manager.cc:1428] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 01:00:35.408277 7936 tablet_bootstrap.cc:492] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: No bootstrap required, opened a new log
I20260806 01:00:35.408411 7936 ts_tablet_manager.cc:1397] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.003s
I20260806 01:00:35.409230 7936 raft_consensus.cc:348] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.409454 7936 raft_consensus.cc:374] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:35.409551 7936 raft_consensus.cc:729] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a4f677190f480ba7fed2d6451f7ce0, State: Initialized, Role: FOLLOWER
I20260806 01:00:35.409792 7936 consensus_queue.cc:260] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.410365 7936 ts_tablet_manager.cc:1428] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 01:00:35.469192 7950 raft_consensus.cc:482] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:35.469425 7950 raft_consensus.cc:504] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.470141 7950 leader_election.cc:288] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5609ef976d46425f9aa8709c1fd0809d (127.5.108.129:36103), 32db6efb919b4fc8aadf9d3060c3eb13 (127.5.108.131:44181)
I20260806 01:00:35.470686 7421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9b3e5f654f4da9b6b08db96ff4d30b" candidate_uuid: "62a4f677190f480ba7fed2d6451f7ce0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5609ef976d46425f9aa8709c1fd0809d" is_pre_election: true
I20260806 01:00:35.470803 7841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9b3e5f654f4da9b6b08db96ff4d30b" candidate_uuid: "62a4f677190f480ba7fed2d6451f7ce0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" is_pre_election: true
I20260806 01:00:35.471006 7421 raft_consensus.cc:2454] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a4f677190f480ba7fed2d6451f7ce0 in term 0.
I20260806 01:00:35.471071 7841 raft_consensus.cc:2454] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a4f677190f480ba7fed2d6451f7ce0 in term 0.
I20260806 01:00:35.471540 7531 leader_election.cc:302] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5609ef976d46425f9aa8709c1fd0809d, 62a4f677190f480ba7fed2d6451f7ce0; no voters:
I20260806 01:00:35.471851 7950 raft_consensus.cc:2781] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:00:35.472007 7950 raft_consensus.cc:482] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:00:35.472148 7950 raft_consensus.cc:3037] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:35.474573 7950 raft_consensus.cc:504] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.475296 7950 leader_election.cc:288] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [CANDIDATE]: Term 1 election: Requested vote from peers 5609ef976d46425f9aa8709c1fd0809d (127.5.108.129:36103), 32db6efb919b4fc8aadf9d3060c3eb13 (127.5.108.131:44181)
I20260806 01:00:35.475858 7841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9b3e5f654f4da9b6b08db96ff4d30b" candidate_uuid: "62a4f677190f480ba7fed2d6451f7ce0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32db6efb919b4fc8aadf9d3060c3eb13"
I20260806 01:00:35.475849 7421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9b3e5f654f4da9b6b08db96ff4d30b" candidate_uuid: "62a4f677190f480ba7fed2d6451f7ce0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5609ef976d46425f9aa8709c1fd0809d"
I20260806 01:00:35.476091 7841 raft_consensus.cc:3037] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:35.476089 7421 raft_consensus.cc:3037] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:35.478142 7421 raft_consensus.cc:2454] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a4f677190f480ba7fed2d6451f7ce0 in term 1.
I20260806 01:00:35.478570 7531 leader_election.cc:302] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5609ef976d46425f9aa8709c1fd0809d, 62a4f677190f480ba7fed2d6451f7ce0; no voters:
I20260806 01:00:35.478858 7950 raft_consensus.cc:2781] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:35.479039 7950 raft_consensus.cc:686] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 LEADER]: Becoming Leader. State: Replica: 62a4f677190f480ba7fed2d6451f7ce0, State: Running, Role: LEADER
I20260806 01:00:35.479068 7841 raft_consensus.cc:2454] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a4f677190f480ba7fed2d6451f7ce0 in term 1.
I20260806 01:00:35.479393 7950 consensus_queue.cc:237] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.482564 7188 catalog_manager.cc:5495] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a4f677190f480ba7fed2d6451f7ce0 (127.5.108.130). New cstate: current_term: 1 leader_uuid: "62a4f677190f480ba7fed2d6451f7ce0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } health_report { overall_health: UNKNOWN } } }
Aug 06 01:00:35 dist-test-slave-x4qh krb5kdc[7122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978033, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:35.552485 7984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.530127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45091 (local address 127.0.0.1:45594)
0806 01:00:35.530775 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.530793 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.530923 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.531322 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.531326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.532257 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:35.532591 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.532603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.534042 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.534047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.535676 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.535690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.536720 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.536726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.536920 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.542678 (+ 5758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:35.542700 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:35.545668 (+ 2968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:35.549354 (+ 3686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:35.549381 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:35.549390 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:35.549981 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:35.550401 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:35.550406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:35.550411 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:35.550602 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:35.551251 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:35.551260 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:35.551522 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.551699 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.552003 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":334,"threads_started":1}
I20260806 01:00:35.552506 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.530009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45594 (local address 127.5.108.190:45091)
0806 01:00:35.530174 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.530180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.530887 (+ 707us) server_negotiation.cc:407] Connection header received
0806 01:00:35.531040 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.531046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.531141 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.531251 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:35.532751 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.533906 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.535846 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.536536 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.536660 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.545878 (+ 9218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:35.545885 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:35.545889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:35.545926 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:35.549214 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.550119 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.550125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.550129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.550284 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.550753 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.550758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.550761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.551109 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:35.551245 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.551837 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.552164 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":55}
I20260806 01:00:35.558396 7188 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:45594:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 01:00:35.559674 7188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:00:35.566565 7391 tablet_service.cc:1467] Processing CreateTablet for tablet e6ef80b28df74093bb7bde6c7e25455f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2e16467e80d34705943f614b741a3ad0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:00:35.566613 7602 tablet_service.cc:1467] Processing CreateTablet for tablet e6ef80b28df74093bb7bde6c7e25455f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2e16467e80d34705943f614b741a3ad0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:00:35.567121 7391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6ef80b28df74093bb7bde6c7e25455f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:35.566947 7811 tablet_service.cc:1467] Processing CreateTablet for tablet e6ef80b28df74093bb7bde6c7e25455f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2e16467e80d34705943f614b741a3ad0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:00:35.567198 7602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6ef80b28df74093bb7bde6c7e25455f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:35.567507 7811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6ef80b28df74093bb7bde6c7e25455f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:35.572713 7937 tablet_bootstrap.cc:492] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: Bootstrap starting.
I20260806 01:00:35.574342 7939 tablet_bootstrap.cc:492] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: Bootstrap starting.
I20260806 01:00:35.574446 7937 tablet_bootstrap.cc:654] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:35.575295 7936 tablet_bootstrap.cc:492] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: Bootstrap starting.
I20260806 01:00:35.576007 7939 tablet_bootstrap.cc:654] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:35.577157 7936 tablet_bootstrap.cc:654] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:35.586113 7939 tablet_bootstrap.cc:492] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: No bootstrap required, opened a new log
I20260806 01:00:35.586122 7937 tablet_bootstrap.cc:492] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: No bootstrap required, opened a new log
I20260806 01:00:35.586283 7937 ts_tablet_manager.cc:1397] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260806 01:00:35.586318 7939 ts_tablet_manager.cc:1397] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260806 01:00:35.587071 7937 raft_consensus.cc:348] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.587324 7937 raft_consensus.cc:374] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:35.587430 7937 raft_consensus.cc:729] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5609ef976d46425f9aa8709c1fd0809d, State: Initialized, Role: FOLLOWER
I20260806 01:00:35.587261 7939 raft_consensus.cc:348] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.587586 7939 raft_consensus.cc:374] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:35.587680 7939 raft_consensus.cc:729] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32db6efb919b4fc8aadf9d3060c3eb13, State: Initialized, Role: FOLLOWER
I20260806 01:00:35.587720 7937 consensus_queue.cc:260] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.588261 7943 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.588653 7943 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:35.588971 7936 tablet_bootstrap.cc:492] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: No bootstrap required, opened a new log
I20260806 01:00:35.589133 7936 ts_tablet_manager.cc:1397] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.009s
I20260806 01:00:35.589138 7937 ts_tablet_manager.cc:1428] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 01:00:35.587960 7939 consensus_queue.cc:260] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.589820 7945 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.590139 7945 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:35.590566 7939 ts_tablet_manager.cc:1428] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 01:00:35.591325 7936 raft_consensus.cc:348] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.591636 7936 raft_consensus.cc:374] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:35.591729 7936 raft_consensus.cc:729] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a4f677190f480ba7fed2d6451f7ce0, State: Initialized, Role: FOLLOWER
I20260806 01:00:35.592063 7936 consensus_queue.cc:260] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.592499 7950 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.592864 7950 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:35.593505 7936 ts_tablet_manager.cc:1428] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260806 01:00:35.634398 7945 raft_consensus.cc:482] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:35.634665 7945 raft_consensus.cc:504] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.636567 7945 leader_election.cc:288] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5609ef976d46425f9aa8709c1fd0809d (127.5.108.129:36103), 62a4f677190f480ba7fed2d6451f7ce0 (127.5.108.130:44833)
I20260806 01:00:35.644647 7943 raft_consensus.cc:482] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:35.644882 7943 raft_consensus.cc:504] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.646790 7943 leader_election.cc:288] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62a4f677190f480ba7fed2d6451f7ce0 (127.5.108.130:44833), 32db6efb919b4fc8aadf9d3060c3eb13 (127.5.108.131:44181)
I20260806 01:00:35.648437 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.637953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:54967 (local address 127.5.108.130:44833)
0806 01:00:35.638145 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.638150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.638933 (+ 783us) server_negotiation.cc:407] Connection header received
0806 01:00:35.639123 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.639128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.639229 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.639313 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:35.640265 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.641499 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.645243 (+ 3744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.647114 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.647232 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.647305 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.647903 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.648077 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:00:35.649730 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.637592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44833 (local address 127.5.108.131:54967)
0806 01:00:35.638721 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.638752 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.638972 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.639353 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.639361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.639419 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:35.639702 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.639719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.642415 (+ 2696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.642424 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.644971 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.644991 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.647485 (+ 2494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.647493 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.647685 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.647720 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.649080 (+ 1360us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.649261 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"mutex_wait_us":51,"thread_start_us":681,"threads_started":1}
I20260806 01:00:35.651410 7636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ef80b28df74093bb7bde6c7e25455f" candidate_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a4f677190f480ba7fed2d6451f7ce0" is_pre_election: true
I20260806 01:00:35.651937 7636 raft_consensus.cc:2454] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32db6efb919b4fc8aadf9d3060c3eb13 in term 0.
I20260806 01:00:35.652712 7736 leader_election.cc:302] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32db6efb919b4fc8aadf9d3060c3eb13, 62a4f677190f480ba7fed2d6451f7ce0; no voters:
I20260806 01:00:35.653121 7945 raft_consensus.cc:2781] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:00:35.653244 7945 raft_consensus.cc:482] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:00:35.653328 7945 raft_consensus.cc:3037] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:35.656666 7945 raft_consensus.cc:504] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.659209 7636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ef80b28df74093bb7bde6c7e25455f" candidate_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a4f677190f480ba7fed2d6451f7ce0"
I20260806 01:00:35.659463 7636 raft_consensus.cc:3037] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:35.661852 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.647721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:56773 (local address 127.5.108.130:44833)
0806 01:00:35.649030 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.649039 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.652385 (+ 3346us) server_negotiation.cc:407] Connection header received
0806 01:00:35.652696 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.652704 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.652809 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.652882 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:35.653958 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.655210 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.658780 (+ 3570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.660349 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.660504 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.660587 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.661329 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.661493 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1162,"thread_start_us":4565,"threads_started":1}
I20260806 01:00:35.662674 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.647261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44833 (local address 127.5.108.129:56773)
0806 01:00:35.652280 (+ 5019us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.652308 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.652502 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.653109 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.653117 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.653156 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:35.653667 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.653684 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.655505 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.655513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.658521 (+ 3008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.658542 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.660896 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.660903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.661099 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.661137 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.662033 (+ 896us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.662239 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4875,"thread_start_us":4676,"threads_started":1}
I20260806 01:00:35.662937 7636 raft_consensus.cc:2454] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32db6efb919b4fc8aadf9d3060c3eb13 in term 1.
I20260806 01:00:35.662989 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.637196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:42293 (local address 127.5.108.129:36103)
0806 01:00:35.637410 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.637417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.637858 (+ 441us) server_negotiation.cc:407] Connection header received
0806 01:00:35.638088 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.638094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.638189 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.638288 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:35.640616 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.642126 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.645389 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.646721 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.646887 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.646976 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.662569 (+ 15593us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.662719 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 01:00:35.663525 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ef80b28df74093bb7bde6c7e25455f" candidate_uuid: "5609ef976d46425f9aa8709c1fd0809d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a4f677190f480ba7fed2d6451f7ce0" is_pre_election: true
I20260806 01:00:35.663753 7736 leader_election.cc:302] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32db6efb919b4fc8aadf9d3060c3eb13, 62a4f677190f480ba7fed2d6451f7ce0; no voters:
I20260806 01:00:35.663892 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.637104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:36103 (local address 127.5.108.131:42293)
0806 01:00:35.637748 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.637771 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.637935 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.640165 (+ 2230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.640170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.640202 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:35.640449 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.640456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.643106 (+ 2650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.643111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.645186 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.645199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.659638 (+ 14439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.659646 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.659838 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.659874 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.663395 (+ 3521us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.663542 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":1046,"threads_started":1}
I20260806 01:00:35.664052 7635 raft_consensus.cc:2379] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5609ef976d46425f9aa8709c1fd0809d in current term 1: Already voted for candidate 32db6efb919b4fc8aadf9d3060c3eb13 in this term.
I20260806 01:00:35.666200 7945 leader_election.cc:288] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [CANDIDATE]: Term 1 election: Requested vote from peers 5609ef976d46425f9aa8709c1fd0809d (127.5.108.129:36103), 62a4f677190f480ba7fed2d6451f7ce0 (127.5.108.130:44833)
I20260806 01:00:35.666662 7945 raft_consensus.cc:2781] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:35.668653 7945 raft_consensus.cc:686] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 LEADER]: Becoming Leader. State: Replica: 32db6efb919b4fc8aadf9d3060c3eb13, State: Running, Role: LEADER
I20260806 01:00:35.669340 7945 consensus_queue.cc:237] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } }
I20260806 01:00:35.670408 7421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ef80b28df74093bb7bde6c7e25455f" candidate_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5609ef976d46425f9aa8709c1fd0809d" is_pre_election: true
I20260806 01:00:35.670665 7421 raft_consensus.cc:2454] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32db6efb919b4fc8aadf9d3060c3eb13 in term 0.
I20260806 01:00:35.671200 7421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ef80b28df74093bb7bde6c7e25455f" candidate_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5609ef976d46425f9aa8709c1fd0809d"
I20260806 01:00:35.671492 7421 raft_consensus.cc:3037] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:35.672072 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.647698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:46355 (local address 127.5.108.131:44181)
0806 01:00:35.647869 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.647875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.663146 (+ 15271us) server_negotiation.cc:407] Connection header received
0806 01:00:35.663426 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.663430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.663495 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.663546 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:35.664213 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.665743 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.669514 (+ 3771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.670845 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.670972 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.671041 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.671653 (+ 612us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.671811 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:00:35.672719 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.647356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:44181 (local address 127.5.108.129:46355)
0806 01:00:35.663060 (+ 15704us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.663078 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.663247 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.663825 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.663830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.663864 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:35.664035 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.664051 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.665940 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.665945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.669276 (+ 3331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.669305 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.671173 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.671180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.671436 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.671484 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.672220 (+ 736us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.672393 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15574,"thread_start_us":22501,"threads_started":1}
I20260806 01:00:35.674290 7421 raft_consensus.cc:2454] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32db6efb919b4fc8aadf9d3060c3eb13 in term 1.
I20260806 01:00:35.675570 7945 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: TxnStatusTablet state changed. Reason: New leader 32db6efb919b4fc8aadf9d3060c3eb13. Latest consensus state: current_term: 1 leader_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.676283 7945 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:00:35.677278 8005 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 01:00:35.677553 8005 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 01:00:35.683301 7841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ef80b28df74093bb7bde6c7e25455f" candidate_uuid: "5609ef976d46425f9aa8709c1fd0809d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" is_pre_election: true
I20260806 01:00:35.683640 7188 catalog_manager.cc:5495] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32db6efb919b4fc8aadf9d3060c3eb13 (127.5.108.131). New cstate: current_term: 1 leader_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } health_report { overall_health: HEALTHY } } }
I20260806 01:00:35.685441 7319 leader_election.cc:302] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5609ef976d46425f9aa8709c1fd0809d; no voters: 32db6efb919b4fc8aadf9d3060c3eb13, 62a4f677190f480ba7fed2d6451f7ce0
I20260806 01:00:35.685884 7943 raft_consensus.cc:2726] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 01:00:35.720175 7984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.711101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:44181 (local address 127.0.0.1:54470)
0806 01:00:35.711411 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.711435 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.711579 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.712436 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.712443 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.712463 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:00:35.712705 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.712720 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.714505 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.714510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.716442 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.716462 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.717792 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.717812 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.718075 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.718164 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:00:35.719435 (+ 1271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:00:35.719471 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.719568 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.719754 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 01:00:35.721004 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.711578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54470 (local address 127.5.108.131:44181)
0806 01:00:35.711772 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.711781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.711813 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:00:35.711924 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.711932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.712060 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.712218 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:35.712933 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.714314 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.716650 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.717506 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.717646 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.718425 (+ 779us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:35.719136 (+ 711us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:35.719294 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.720531 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.720633 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:00:35.726554 7421 raft_consensus.cc:1261] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Refusing update from remote peer 32db6efb919b4fc8aadf9d3060c3eb13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:35.726689 7636 raft_consensus.cc:1261] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 FOLLOWER]: Refusing update from remote peer 32db6efb919b4fc8aadf9d3060c3eb13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:35.727321 7997 consensus_queue.cc:1035] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:35.728259 7945 consensus_queue.cc:1035] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:35.732445 7943 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: TxnStatusTablet state changed. Reason: New leader 32db6efb919b4fc8aadf9d3060c3eb13. Latest consensus state: current_term: 1 leader_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.732761 7943 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:35.734898 7945 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.735159 7945 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:00:35.741698 7955 mvcc.cc:205] Tried to move back new op lower bound from 7315366034327789568 to 7315366034116874240. Current Snapshot: MvccSnapshot[applied={T|T < 7315366034327789568 or (T in {7315366034327789568})}]
I20260806 01:00:35.742478 7943 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.742718 7943 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 5609ef976d46425f9aa8709c1fd0809d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:35.750825 7997 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.751214 7997 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 32db6efb919b4fc8aadf9d3060c3eb13: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:00:35.752282 7950 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: TxnStatusTablet state changed. Reason: New leader 32db6efb919b4fc8aadf9d3060c3eb13. Latest consensus state: current_term: 1 leader_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.752544 7950 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:35.754132 7950 tablet_replica.cc:439] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 } } peers { permanent_uuid: "62a4f677190f480ba7fed2d6451f7ce0" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44833 } } peers { permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 } } }
I20260806 01:00:35.754449 7950 tablet_replica.cc:442] T e6ef80b28df74093bb7bde6c7e25455f P 62a4f677190f480ba7fed2d6451f7ce0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:35.774682 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.768707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:46501 (local address 127.5.108.131:44181)
0806 01:00:35.768896 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.768909 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.769529 (+ 620us) server_negotiation.cc:407] Connection header received
0806 01:00:35.769744 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.769750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.769817 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.769913 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:35.770396 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.771139 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.772708 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.773214 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.773351 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.773826 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:35.773922 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:35.774023 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.774175 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.774267 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:00:35.774710 8014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.768679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:44181 (local address 127.5.108.130:46501)
0806 01:00:35.769428 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.769443 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.769614 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.769929 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.769933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.769944 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:00:35.770232 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.770242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.771294 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.771299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.772553 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.772566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.773405 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.773413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.773558 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.773632 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:00:35.774023 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:00:35.774036 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.774148 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.774274 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":422,"threads_started":1}
I20260806 01:00:35.790191 7421 raft_consensus.cc:1261] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Refusing update from remote peer 62a4f677190f480ba7fed2d6451f7ce0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:35.790382 7841 raft_consensus.cc:1261] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Refusing update from remote peer 62a4f677190f480ba7fed2d6451f7ce0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:35.790751 7944 consensus_queue.cc:1035] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5609ef976d46425f9aa8709c1fd0809d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:35.790992 7950 consensus_queue.cc:1035] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32db6efb919b4fc8aadf9d3060c3eb13" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:35.798594 7953 mvcc.cc:205] Tried to move back new op lower bound from 7315366034589208576 to 7315366033326440448. Current Snapshot: MvccSnapshot[applied={T|T < 7315366034589208576}]
I20260806 01:00:35.848613 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.839690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:59201 (local address 127.5.108.130:44833)
0806 01:00:35.839920 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.839929 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.842111 (+ 2182us) server_negotiation.cc:407] Connection header received
0806 01:00:35.842352 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.842357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.842436 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.842554 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:35.843070 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.843965 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.846394 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.846973 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.847049 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.847679 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:35.847816 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:35.847893 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.848207 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.848292 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:00:35.849442 8019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.839697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44833 (local address 127.5.108.131:59201)
0806 01:00:35.841997 (+ 2300us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:35.842018 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:35.842200 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:35.842568 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:35.842572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:35.842584 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:00:35.842918 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.842927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.844098 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.844110 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.846211 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:35.846226 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.847217 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.847221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:35.847346 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.847467 (+ 121us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:00:35.847977 (+ 510us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:00:35.847996 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 01:00:35.848874 (+ 878us) client_negotiation.cc:240] Negotiation successful
0806 01:00:35.849011 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2106,"thread_start_us":1950,"threads_started":1}
I20260806 01:00:35.890338 7192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:35.890594 7192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:35.893721 7192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b23e20e061a4cdfbabcb40fe5c74e42: Sending DeleteTablet for 3 replicas of tablet fe9b3e5f654f4da9b6b08db96ff4d30b
I20260806 01:00:35.894327 7391 tablet_service.cc:1514] Processing DeleteTablet for tablet fe9b3e5f654f4da9b6b08db96ff4d30b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:35 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:48376
I20260806 01:00:35.894761 7811 tablet_service.cc:1514] Processing DeleteTablet for tablet fe9b3e5f654f4da9b6b08db96ff4d30b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:35 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:54466
I20260806 01:00:35.895030 7964 tablet_replica.cc:330] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: stopping tablet replica
I20260806 01:00:35.895030 7602 tablet_service.cc:1514] Processing DeleteTablet for tablet fe9b3e5f654f4da9b6b08db96ff4d30b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:35 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:38910
I20260806 01:00:35.895234 7964 raft_consensus.cc:2229] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:35.895495 7964 raft_consensus.cc:2258] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:35.897027 7964 ts_tablet_manager.cc:1905] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:35.906330 7964 ts_tablet_manager.cc:1918] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:00:35.906519 7964 log.cc:1196] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/fe9b3e5f654f4da9b6b08db96ff4d30b
I20260806 01:00:35.907018 7964 ts_tablet_manager.cc:1939] T fe9b3e5f654f4da9b6b08db96ff4d30b P 32db6efb919b4fc8aadf9d3060c3eb13: Deleting consensus metadata
I20260806 01:00:35.916888 7163 catalog_manager.cc:4841] TS 32db6efb919b4fc8aadf9d3060c3eb13 (127.5.108.131:44181): tablet fe9b3e5f654f4da9b6b08db96ff4d30b (table test-table [id=6f19cb6801b94e9c8559b3b60e1e6ab1]) successfully deleted
I20260806 01:00:35.919912 8024 tablet_replica.cc:330] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: stopping tablet replica
I20260806 01:00:35.920316 8024 raft_consensus.cc:2229] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:35.920697 8024 raft_consensus.cc:2258] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:35.922351 8024 ts_tablet_manager.cc:1905] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:35.939643 8026 tablet_replica.cc:330] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: stopping tablet replica
I20260806 01:00:35.939877 8026 raft_consensus.cc:2229] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:00:35.940234 8026 raft_consensus.cc:2258] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:35.941803 8026 ts_tablet_manager.cc:1905] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:00:35 dist-test-slave-x4qh krb5kdc[7122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978035, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:00:35.948990 8024 ts_tablet_manager.cc:1918] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:00:35.949237 8024 log.cc:1196] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/fe9b3e5f654f4da9b6b08db96ff4d30b
I20260806 01:00:35.949774 8024 ts_tablet_manager.cc:1939] T fe9b3e5f654f4da9b6b08db96ff4d30b P 5609ef976d46425f9aa8709c1fd0809d: Deleting consensus metadata
I20260806 01:00:35.951170 7168 catalog_manager.cc:4841] TS 5609ef976d46425f9aa8709c1fd0809d (127.5.108.129:36103): tablet fe9b3e5f654f4da9b6b08db96ff4d30b (table test-table [id=6f19cb6801b94e9c8559b3b60e1e6ab1]) successfully deleted
I20260806 01:00:35.953912 8026 ts_tablet_manager.cc:1918] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:00:35.954080 8026 log.cc:1196] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/fe9b3e5f654f4da9b6b08db96ff4d30b
I20260806 01:00:35.954666 8026 ts_tablet_manager.cc:1939] T fe9b3e5f654f4da9b6b08db96ff4d30b P 62a4f677190f480ba7fed2d6451f7ce0: Deleting consensus metadata
I20260806 01:00:35.955875 7164 catalog_manager.cc:4841] TS 62a4f677190f480ba7fed2d6451f7ce0 (127.5.108.130:44833): tablet fe9b3e5f654f4da9b6b08db96ff4d30b (table test-table [id=6f19cb6801b94e9c8559b3b60e1e6ab1]) successfully deleted
I20260806 01:00:35.956640 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.918677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.5.108.129:36103)
0806 01:00:35.918900 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.918908 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.921384 (+ 2476us) server_negotiation.cc:407] Connection header received
0806 01:00:35.922848 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.922856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.922968 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.923053 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:35.935512 (+ 12459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.936827 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.940510 (+ 3683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.940979 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.941059 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.948707 (+ 7648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:35.948726 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:35.948748 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:35.948791 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:35.952605 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.953574 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.953586 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.953592 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.953789 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.954456 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.954464 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.954470 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.955058 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:35.955409 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.956255 (+ 846us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.956419 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 01:00:35.958104 7487 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48390
I20260806 01:00:35.974020 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.961454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45600 (local address 127.5.108.190:45091)
0806 01:00:35.961734 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.961743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.962184 (+ 441us) server_negotiation.cc:407] Connection header received
0806 01:00:35.962370 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.962376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.962454 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.962564 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:35.963940 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.964663 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.965931 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.966330 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.966461 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.968675 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:35.968681 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:35.968692 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:35.968724 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:35.971323 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.972049 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.972056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.972059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.972204 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.972610 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.972614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.972618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.972893 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:35.973201 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.973481 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.973706 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 01:00:35.974601 7192 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45600
I20260806 01:00:35.976428 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 7291
I20260806 01:00:35.989615 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:35.977223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48406 (local address 127.5.108.129:36103)
0806 01:00:35.977415 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:35.977423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:35.977865 (+ 442us) server_negotiation.cc:407] Connection header received
0806 01:00:35.978047 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:35.978052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:35.978127 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:35.978241 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:35.979296 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.980064 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.981250 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:35.981716 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:35.981859 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:35.984172 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:35.984178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:35.984184 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:35.984220 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:35.986936 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.987675 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.987682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.987686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.987817 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:35.988190 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:35.988195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:35.988198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:35.988462 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:35.988754 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:35.989047 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 01:00:35.989272 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 01:00:35.990221 7487 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48406
W20260806 01:00:35.990803 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 7291: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:00:35.990969 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 7291
I20260806 01:00:36.025440 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 7502
Aug 06 01:00:36 dist-test-slave-x4qh krb5kdc[7122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978035, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:00:36.042291 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:36.026610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38928 (local address 127.5.108.130:44833)
0806 01:00:36.026827 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:36.026837 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:36.026860 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:00:36.026935 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:36.026941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:36.027031 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:36.027113 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:36.029078 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:36.029809 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:36.031661 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:36.032080 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:36.032221 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:36.036582 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:36.036596 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:36.036609 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:36.036640 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:36.039418 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:36.040198 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:36.040205 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:36.040208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:36.040350 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:36.040740 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:36.040744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:36.040746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:36.041068 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:36.041387 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:36.041689 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 01:00:36.041909 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:00:36.043324 7698 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38928
W20260806 01:00:36.044073 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 7502: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:00:36.044265 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 7502
I20260806 01:00:36.074447 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 7709
Aug 06 01:00:36 dist-test-slave-x4qh krb5kdc[7122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978035, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:00:36.090577 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:36.075166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.5.108.131:44181)
0806 01:00:36.075349 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:36.075355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:36.075527 (+ 172us) server_negotiation.cc:407] Connection header received
0806 01:00:36.075745 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:36.075749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:36.075808 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:36.075926 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:36.077114 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:36.077891 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:36.079051 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:36.079665 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:36.079797 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:36.083969 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:36.083988 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:36.084004 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:36.084049 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:36.087818 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:36.088427 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:36.088435 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:36.088439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:36.088601 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:36.088958 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:36.088963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:36.088967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:36.089449 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:36.089808 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:36.089971 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 01:00:36.090180 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 01:00:36.091593 7899 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54480
W20260806 01:00:36.092222 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 7709: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:00:36.092338 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 7709
I20260806 01:00:36.121696 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 7141
I20260806 01:00:36.122795 7247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:36.234223 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 7141
2026-08-06T01:00:36Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4688 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:00:36 dist-test-slave-x4qh krb5kdc[8040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:00:36 dist-test-slave-x4qh krb5kdc[8040](info): set up 2 sockets
Aug 06 01:00:36 dist-test-slave-x4qh krb5kdc[8040](info): commencing operation
krb5kdc: starting...
W20260806 01:00:37.473011 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:00:37 dist-test-slave-x4qh krb5kdc[8040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:00:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:00:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:00:37.662787 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:46751
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:46751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:37.968611 8061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:37.968894 8061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:37.968981 8061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:37.975752 8061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:00:37.975837 8061 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 01:00:37.975881 8061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:37.975940 8061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:37.975987 8061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:00:37.976043 8061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:00:37.985402 8061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:46751
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:46751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:37.986550 8061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:37.987820 8061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:37.999329 8072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:37.999428 8069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:37.999465 8068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:38.000598 8061 server_base.cc:1034] running on GCE node
I20260806 01:00:38.001379 8061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:38.002683 8061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:38.003927 8061 hybrid_clock.cc:648] HybridClock initialized: now 1785978038003863 us; error 49 us; skew 500 ppm
I20260806 01:00:38.004227 8061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:38 dist-test-slave-x4qh krb5kdc[8040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:38.007351 8061 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:00:38.008688 8061 webserver.cc:460] Webserver started at http://127.5.108.190:34291/ using document root <none> and password file <none>
I20260806 01:00:38.009348 8061 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:38.009473 8061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:38.009773 8061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:38.012610 8061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "c846a7944ebe4a8dad7a78e6b8a981ac"
format_stamp: "Formatted at 2026-08-06 01:00:38 on dist-test-slave-x4qh"
server_key: "c72d16babd6732826164035912adba58"
server_key_iv: "1d9c1e2922daa116a5df57b3f217b892"
server_key_version: "encryptionkey@0"
I20260806 01:00:38.013381 8061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "c846a7944ebe4a8dad7a78e6b8a981ac"
format_stamp: "Formatted at 2026-08-06 01:00:38 on dist-test-slave-x4qh"
server_key: "c72d16babd6732826164035912adba58"
server_key_iv: "1d9c1e2922daa116a5df57b3f217b892"
server_key_version: "encryptionkey@0"
I20260806 01:00:38.018030 8061 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:00:38.021082 8084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:38.022303 8061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 01:00:38.022439 8061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "c846a7944ebe4a8dad7a78e6b8a981ac"
format_stamp: "Formatted at 2026-08-06 01:00:38 on dist-test-slave-x4qh"
server_key: "c72d16babd6732826164035912adba58"
server_key_iv: "1d9c1e2922daa116a5df57b3f217b892"
server_key_version: "encryptionkey@0"
I20260806 01:00:38.022603 8061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:38.038166 8061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:38.041682 8061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:38.041894 8061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:38.063210 8061 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:46751
I20260806 01:00:38.063256 8184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:46751 every 8 connection(s)
I20260806 01:00:38.064488 8061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:00:38.067919 8186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:38.068638 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 8061
I20260806 01:00:38.068871 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:00:38.069744 5554 external_mini_cluster.cc:1428] Setting key ed073c90974d18a84b4e297338879072
I20260806 01:00:38.082057 8186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac: Bootstrap starting.
Aug 06 01:00:38 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:38.085502 8186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:38.086643 8186 log.cc:824] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:38.089316 8186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac: No bootstrap required, opened a new log
I20260806 01:00:38.092561 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:38.071768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.5.108.190:46751)
0806 01:00:38.072543 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:38.072554 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:38.072595 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:00:38.074394 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:38.074415 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:38.074763 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:38.075111 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:38.076942 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.078145 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.079928 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.080567 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.080699 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:38.085362 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:38.085369 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:38.085382 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:38.085417 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:38.088771 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:38.089394 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:38.089403 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:38.089412 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:38.089601 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:38.089946 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:38.089950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:38.089953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:38.090449 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:38.090750 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:38.091192 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 01:00:38.091639 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":370,"threads_started":1}
I20260806 01:00:38.096948 8186 raft_consensus.cc:348] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c846a7944ebe4a8dad7a78e6b8a981ac" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46751 } }
I20260806 01:00:38.097294 8186 raft_consensus.cc:374] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:38.097416 8186 raft_consensus.cc:729] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c846a7944ebe4a8dad7a78e6b8a981ac, State: Initialized, Role: FOLLOWER
I20260806 01:00:38.097970 8186 consensus_queue.cc:260] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c846a7944ebe4a8dad7a78e6b8a981ac" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46751 } }
I20260806 01:00:38.098222 8186 raft_consensus.cc:388] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:00:38.098356 8186 raft_consensus.cc:482] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:00:38.098523 8186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:38.100881 8186 raft_consensus.cc:504] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c846a7944ebe4a8dad7a78e6b8a981ac" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46751 } }
I20260806 01:00:38.101372 8186 leader_election.cc:302] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c846a7944ebe4a8dad7a78e6b8a981ac; no voters:
I20260806 01:00:38.101924 8186 leader_election.cc:288] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:00:38.102017 8192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:38.102768 8192 raft_consensus.cc:686] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [term 1 LEADER]: Becoming Leader. State: Replica: c846a7944ebe4a8dad7a78e6b8a981ac, State: Running, Role: LEADER
I20260806 01:00:38.103255 8192 consensus_queue.cc:237] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c846a7944ebe4a8dad7a78e6b8a981ac" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46751 } }
I20260806 01:00:38.103986 8186 sys_catalog.cc:564] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:00:38.105895 8193 sys_catalog.cc:455] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c846a7944ebe4a8dad7a78e6b8a981ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c846a7944ebe4a8dad7a78e6b8a981ac" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46751 } } }
I20260806 01:00:38.106210 8193 sys_catalog.cc:458] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:38.107033 8198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:00:38.107759 8194 sys_catalog.cc:455] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader c846a7944ebe4a8dad7a78e6b8a981ac. Latest consensus state: current_term: 1 leader_uuid: "c846a7944ebe4a8dad7a78e6b8a981ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c846a7944ebe4a8dad7a78e6b8a981ac" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46751 } } }
I20260806 01:00:38.108036 8194 sys_catalog.cc:458] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:38.112385 8198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:00:38.122583 8198 catalog_manager.cc:1347] Generated new cluster ID: a6b6e46c8c304aa78075a3d78c1e8232
I20260806 01:00:38.122689 8198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:00:38.151829 8198 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:00:38.153017 8198 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:00:38.162803 8198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac: Generated new TSK 0
I20260806 01:00:38.163552 8198 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:00:38.224690 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46751
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:38.563709 8219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:38.564036 8219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:38.564239 8219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:38.571278 8219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:38.571453 8219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:38.571660 8219 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 01:00:38.571763 8219 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:00:38.571892 8219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:00:38.581020 8219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.108.190:46751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:38.582379 8219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:38.583794 8219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:38.597904 8226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:38.597868 8231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:38.597973 8228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:38.598744 8219 server_base.cc:1034] running on GCE node
I20260806 01:00:38.599304 8219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:38.600179 8219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:38.601374 8219 hybrid_clock.cc:648] HybridClock initialized: now 1785978038601324 us; error 45 us; skew 500 ppm
I20260806 01:00:38.601707 8219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:38 dist-test-slave-x4qh krb5kdc[8040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:38.604982 8219 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:00:38.606385 8219 webserver.cc:460] Webserver started at http://127.5.108.129:46005/ using document root <none> and password file <none>
I20260806 01:00:38.606969 8219 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:38.607067 8219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:38.607309 8219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:38.610206 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "129b5aaf37054fc69f7936cf5b70f386"
format_stamp: "Formatted at 2026-08-06 01:00:38 on dist-test-slave-x4qh"
server_key: "ebb051202c7f2f50edbff5e27e6834bc"
server_key_iv: "4c899f5f0730487d53df8e70bc43877d"
server_key_version: "encryptionkey@0"
I20260806 01:00:38.610894 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "129b5aaf37054fc69f7936cf5b70f386"
format_stamp: "Formatted at 2026-08-06 01:00:38 on dist-test-slave-x4qh"
server_key: "ebb051202c7f2f50edbff5e27e6834bc"
server_key_iv: "4c899f5f0730487d53df8e70bc43877d"
server_key_version: "encryptionkey@0"
I20260806 01:00:38.615821 8219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 01:00:38.619053 8239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:38.620343 8219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:00:38.620479 8219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "129b5aaf37054fc69f7936cf5b70f386"
format_stamp: "Formatted at 2026-08-06 01:00:38 on dist-test-slave-x4qh"
server_key: "ebb051202c7f2f50edbff5e27e6834bc"
server_key_iv: "4c899f5f0730487d53df8e70bc43877d"
server_key_version: "encryptionkey@0"
I20260806 01:00:38.620635 8219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:38.640743 8219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:38.644495 8219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:38.644726 8219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:38.647199 8219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:38.647295 8219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:38.647454 8219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:38.647549 8219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:00:38 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:38.671241 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:38.648119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:47117 (local address 127.5.108.190:46751)
0806 01:00:38.648670 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:38.648675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:38.649788 (+ 1113us) server_negotiation.cc:407] Connection header received
0806 01:00:38.651866 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:38.651872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:38.651962 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:38.652092 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:38.654290 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.655023 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.658186 (+ 3163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.658580 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.658681 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:38.663269 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:38.663278 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:38.663286 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:38.663338 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:38.667439 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:38.668569 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:38.668577 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:38.668580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:38.668735 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:38.669375 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:38.669379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:38.669381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:38.669673 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:38.669909 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:38.670588 (+ 679us) server_negotiation.cc:299] Negotiation successful
0806 01:00:38.670871 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":294,"threads_started":1}
I20260806 01:00:38.672284 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:38.648737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46751 (local address 127.5.108.129:47117)
0806 01:00:38.649629 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:38.649665 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:38.651600 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:38.652376 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:38.652391 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:38.653230 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:38.654112 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:38.654134 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.655228 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.655246 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:38.658019 (+ 2773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:38.658035 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.658846 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.658855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:38.659169 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:38.659988 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:38.660019 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:38.662908 (+ 2889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:38.667621 (+ 4713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:38.667633 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:38.667651 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:38.668389 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:38.668879 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:38.668885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:38.668890 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:38.669130 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:38.669876 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:38.669884 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:38.670222 (+ 338us) client_negotiation.cc:769] Sending connection context
0806 01:00:38.670485 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 01:00:38.670988 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":378,"threads_started":1}
I20260806 01:00:38.693643 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:38.678261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:58953 (local address 127.5.108.190:46751)
0806 01:00:38.678465 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:38.678473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:38.678993 (+ 520us) server_negotiation.cc:407] Connection header received
0806 01:00:38.679104 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:38.679112 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:38.679198 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:38.679314 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:38.680630 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.681592 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.683531 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.684250 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.684428 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:38.686443 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:38.686453 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:38.686460 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:38.686507 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:38.690464 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:38.691093 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:38.691100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:38.691105 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:38.691281 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:38.691831 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:38.691838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:38.691842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:38.692226 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:38.692387 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:38.692931 (+ 544us) server_negotiation.cc:299] Negotiation successful
0806 01:00:38.693197 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:00:38.693938 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:38.678262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46751 (local address 127.5.108.129:58953)
0806 01:00:38.678908 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:38.678922 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:38.679020 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:38.679346 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:38.679349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:38.679962 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:38.680476 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:38.680491 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.681778 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.681785 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:38.683345 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:38.683367 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:38.684521 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:38.684529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:38.684749 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:38.685433 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:38.685453 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:38.686238 (+ 785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:38.690603 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:38.690607 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:38.690610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:38.690982 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:38.691458 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:38.691464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:38.691469 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:38.691690 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:38.692401 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:38.692408 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:38.692584 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 01:00:38.692775 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 01:00:38.693182 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":349,"threads_started":1}
I20260806 01:00:38.707487 8219 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:42091
I20260806 01:00:38.707530 8437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:42091 every 8 connection(s)
I20260806 01:00:38.708634 8219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:00:38.710719 8438 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46751
I20260806 01:00:38.710908 8438 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:38.711529 8438 heartbeater.cc:511] Master 127.5.108.190:46751 requested a full tablet report, sending...
I20260806 01:00:38.713280 8104 ts_manager.cc:194] Registered new tserver with Master: 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129:42091)
I20260806 01:00:38.714643 8104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:47117
I20260806 01:00:38.716415 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 8219
I20260806 01:00:38.716660 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:00:38.717494 5554 external_mini_cluster.cc:1428] Setting key c19a7b0a0655057ac795dfc854421e96
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:00:38.760663 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46751
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:39.079221 8447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:39.079556 8447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:39.079735 8447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:39.086552 8447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:39.086692 8447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:39.086823 8447 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 01:00:39.086905 8447 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:00:39.087023 8447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:00:39.095978 8447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.108.190:46751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:39.097162 8447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:39.098323 8447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:39.110992 8456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:39.111101 8462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:39.111250 8458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:39.112481 8447 server_base.cc:1034] running on GCE node
I20260806 01:00:39.113114 8447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:39.114025 8447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:39.115272 8447 hybrid_clock.cc:648] HybridClock initialized: now 1785978039115203 us; error 46 us; skew 500 ppm
I20260806 01:00:39.115638 8447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:39 dist-test-slave-x4qh krb5kdc[8040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978039, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:39.123057 8447 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:00:39.124653 8447 webserver.cc:460] Webserver started at http://127.5.108.130:46033/ using document root <none> and password file <none>
I20260806 01:00:39.125322 8447 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:39.125470 8447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:39.125782 8447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:39.128799 8447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "bd0f29970cc94da4b18a134fb024f348"
format_stamp: "Formatted at 2026-08-06 01:00:39 on dist-test-slave-x4qh"
server_key: "459e165a1cd63439aa82f89962abebdb"
server_key_iv: "6af3fb9f02b0d1d1eef2f9395a89caaa"
server_key_version: "encryptionkey@0"
I20260806 01:00:39.129510 8447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "bd0f29970cc94da4b18a134fb024f348"
format_stamp: "Formatted at 2026-08-06 01:00:39 on dist-test-slave-x4qh"
server_key: "459e165a1cd63439aa82f89962abebdb"
server_key_iv: "6af3fb9f02b0d1d1eef2f9395a89caaa"
server_key_version: "encryptionkey@0"
I20260806 01:00:39.134335 8447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:00:39.137511 8469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:39.138652 8447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 01:00:39.138821 8447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "bd0f29970cc94da4b18a134fb024f348"
format_stamp: "Formatted at 2026-08-06 01:00:39 on dist-test-slave-x4qh"
server_key: "459e165a1cd63439aa82f89962abebdb"
server_key_iv: "6af3fb9f02b0d1d1eef2f9395a89caaa"
server_key_version: "encryptionkey@0"
I20260806 01:00:39.139008 8447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:39.149960 8447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:39.154208 8447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:39.154496 8447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:39.157080 8447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:39.157210 8447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:39.157366 8447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:39.157462 8447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:00:39 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978039, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:39.179572 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.157553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:38843 (local address 127.5.108.190:46751)
0806 01:00:39.157780 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:39.157789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:39.159087 (+ 1298us) server_negotiation.cc:407] Connection header received
0806 01:00:39.161199 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:39.161205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:39.161290 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:39.161400 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:39.163363 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.164330 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.166927 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.167526 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.167640 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.172765 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:39.172774 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:39.172792 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:39.172849 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:39.175917 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.177047 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.177054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.177057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.177203 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.177841 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.177849 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.177853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.178198 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:39.178359 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:39.179012 (+ 653us) server_negotiation.cc:299] Negotiation successful
0806 01:00:39.179221 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:00:39.180699 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.158060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46751 (local address 127.5.108.130:38843)
0806 01:00:39.158908 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:39.158950 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:39.160908 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:39.161672 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:39.161685 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:39.162466 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:39.163206 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.163223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.164521 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.164526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.166694 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.166716 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.167780 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.167790 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.168154 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.168944 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:39.168981 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:39.172539 (+ 3558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:39.176105 (+ 3566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:39.176116 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:39.176129 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:39.176861 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:39.177355 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:39.177361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:39.177367 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:39.177682 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:39.178344 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:39.178351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:39.178665 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 01:00:39.178929 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 01:00:39.179418 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":378,"threads_started":1}
I20260806 01:00:39.201784 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.187402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:39129 (local address 127.5.108.190:46751)
0806 01:00:39.187761 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:39.187767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:39.188311 (+ 544us) server_negotiation.cc:407] Connection header received
0806 01:00:39.188481 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:39.188484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:39.188536 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:39.188610 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:39.190046 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.190740 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.193164 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.193533 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.193618 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.195612 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:39.195616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:39.195619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:39.195645 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:39.198792 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.199491 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.199496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.199500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.199635 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.200072 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.200077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.200081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.200391 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:39.200546 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:39.201347 (+ 801us) server_negotiation.cc:299] Negotiation successful
0806 01:00:39.201553 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260806 01:00:39.201972 8593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.187361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46751 (local address 127.5.108.130:39129)
0806 01:00:39.188191 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:39.188215 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:39.188372 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:39.188691 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:39.188698 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:39.189404 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:39.189892 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.189908 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.190867 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.190874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.193021 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.193039 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.193747 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.193756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.193995 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.194674 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:39.194689 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:39.195423 (+ 734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:39.198914 (+ 3491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:39.198919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:39.198922 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:39.199324 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:39.199783 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:39.199806 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:39.199810 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:39.199944 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:39.200513 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:39.200518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:39.200755 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 01:00:39.200928 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 01:00:39.201323 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":447,"threads_started":1}
I20260806 01:00:39.209851 8447 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:39231
I20260806 01:00:39.209923 8652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:39231 every 8 connection(s)
I20260806 01:00:39.210988 8447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:00:39.213035 8653 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46751
I20260806 01:00:39.213217 8653 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:39.213768 8653 heartbeater.cc:511] Master 127.5.108.190:46751 requested a full tablet report, sending...
I20260806 01:00:39.214921 8104 ts_manager.cc:194] Registered new tserver with Master: bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231)
I20260806 01:00:39.215723 8104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:38843
I20260806 01:00:39.217691 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 8447
I20260806 01:00:39.217835 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:00:39.218220 5554 external_mini_cluster.cc:1428] Setting key 6fb43c7036fc1e1380a8d2b34881c1f1
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:00:39.260303 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46751
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:39.579768 8658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:39.580029 8658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:39.580191 8658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:39.587453 8658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:39.587574 8658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:39.587743 8658 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 01:00:39.587826 8658 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:00:39.587939 8658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:00:39.597218 8658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.108.190:46751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:39.598438 8658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:39.599777 8658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:39.612596 8665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:39.612643 8667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:39.612846 8664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:39.613852 8658 server_base.cc:1034] running on GCE node
I20260806 01:00:39.614506 8658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:39.615309 8658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:39.616560 8658 hybrid_clock.cc:648] HybridClock initialized: now 1785978039616506 us; error 44 us; skew 500 ppm
I20260806 01:00:39.616880 8658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:39 dist-test-slave-x4qh krb5kdc[8040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978039, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:39.619982 8658 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:00:39.621380 8658 webserver.cc:460] Webserver started at http://127.5.108.131:43271/ using document root <none> and password file <none>
I20260806 01:00:39.622031 8658 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:39.622166 8658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:39.622491 8658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:39.625556 8658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "5ed2c8fcac404bf3a78b789a2b277978"
format_stamp: "Formatted at 2026-08-06 01:00:39 on dist-test-slave-x4qh"
server_key: "ef0e830e616cec088b07fa0b9e80e530"
server_key_iv: "22d0912787a594c658640f2ebe6a1478"
server_key_version: "encryptionkey@0"
I20260806 01:00:39.626323 8658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "5ed2c8fcac404bf3a78b789a2b277978"
format_stamp: "Formatted at 2026-08-06 01:00:39 on dist-test-slave-x4qh"
server_key: "ef0e830e616cec088b07fa0b9e80e530"
server_key_iv: "22d0912787a594c658640f2ebe6a1478"
server_key_version: "encryptionkey@0"
I20260806 01:00:39.631177 8658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260806 01:00:39.634619 8677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:39.635983 8658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:00:39.636169 8658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "5ed2c8fcac404bf3a78b789a2b277978"
format_stamp: "Formatted at 2026-08-06 01:00:39 on dist-test-slave-x4qh"
server_key: "ef0e830e616cec088b07fa0b9e80e530"
server_key_iv: "22d0912787a594c658640f2ebe6a1478"
server_key_version: "encryptionkey@0"
I20260806 01:00:39.636361 8658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:39.661720 8658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:39.665432 8658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:39.665756 8658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:39.668434 8658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:39.668571 8658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:39.668717 8658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:39.668807 8658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:00:39 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978039, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:39.695322 8689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.669486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46751 (local address 127.5.108.131:59857)
0806 01:00:39.670344 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:39.670379 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:39.672989 (+ 2610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:39.674055 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:39.674066 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:39.674974 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:39.676144 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.676179 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.677888 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.677896 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.680894 (+ 2998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.680916 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.682004 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.682013 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.682216 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.682952 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:39.682988 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:39.686518 (+ 3530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:39.689989 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:39.690002 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:39.690026 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:39.690846 (+ 820us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:39.691424 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:39.691429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:39.691433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:39.691693 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:39.692477 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:39.692487 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:39.692878 (+ 391us) client_negotiation.cc:769] Sending connection context
0806 01:00:39.693283 (+ 405us) client_negotiation.cc:240] Negotiation successful
0806 01:00:39.693813 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":368,"threads_started":1}
I20260806 01:00:39.696388 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.668889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:59857 (local address 127.5.108.190:46751)
0806 01:00:39.669111 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:39.669120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:39.670490 (+ 1370us) server_negotiation.cc:407] Connection header received
0806 01:00:39.673322 (+ 2832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:39.673332 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:39.673450 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:39.673566 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:39.676379 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.677701 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.681085 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.681634 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.681811 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.686798 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:39.686803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:39.686806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:39.686838 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:39.689839 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.691010 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.691018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.691023 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.691205 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.691858 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.691865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.691869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.692285 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:39.692468 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:39.695549 (+ 3081us) server_negotiation.cc:299] Negotiation successful
0806 01:00:39.695867 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 01:00:39.718837 8438 heartbeater.cc:503] Master 127.5.108.190:46751 was elected leader, sending a full tablet report...
I20260806 01:00:39.722043 8658 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:46841
I20260806 01:00:39.722224 8846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:46841 every 8 connection(s)
I20260806 01:00:39.723446 8658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:00:39.725800 8847 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46751
I20260806 01:00:39.726061 8847 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:39.726750 8847 heartbeater.cc:511] Master 127.5.108.190:46751 requested a full tablet report, sending...
I20260806 01:00:39.727727 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 8658
I20260806 01:00:39.727873 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:00:39.728263 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.707057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:60177 (local address 127.5.108.190:46751)
0806 01:00:39.707243 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:39.707251 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:39.712167 (+ 4916us) server_negotiation.cc:407] Connection header received
0806 01:00:39.712353 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:39.712358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:39.712434 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:39.712532 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:39.714665 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.715782 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.718106 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.718832 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.718955 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.721727 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:39.721734 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:39.721738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:39.721804 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:39.725312 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.726108 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.726114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.726119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.726262 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.726697 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.726703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.726707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.727059 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:39.727223 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:39.727704 (+ 481us) server_negotiation.cc:299] Negotiation successful
0806 01:00:39.727909 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:00:39.728286 5554 external_mini_cluster.cc:1428] Setting key c524a9244b46c622a12dd021b4aacf1a
I20260806 01:00:39.728386 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.706578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46751 (local address 127.5.108.131:60177)
0806 01:00:39.712010 (+ 5432us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:39.712036 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:39.712222 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:39.712705 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:39.712710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:39.713419 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:39.714470 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.714494 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.716013 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.716020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.717856 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.717872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.719272 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.719281 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.719509 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.720239 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:39.720256 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:39.721524 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:39.725517 (+ 3993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:39.725522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:39.725527 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:39.725974 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:39.726377 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:39.726382 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:39.726385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:39.726556 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:39.727189 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:39.727194 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:39.727449 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 01:00:39.727600 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 01:00:39.727909 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5233,"thread_start_us":5040,"threads_started":1}
I20260806 01:00:39.729318 8104 ts_manager.cc:194] Registered new tserver with Master: 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:39.730557 8104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:59857
I20260806 01:00:39.732134 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:00:39 dist-test-slave-x4qh krb5kdc[8040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:00:39 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:39.794461 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.780403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35778 (local address 127.5.108.190:46751)
0806 01:00:39.780651 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:39.780658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:39.781092 (+ 434us) server_negotiation.cc:407] Connection header received
0806 01:00:39.781271 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:39.781279 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:39.781405 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:39.781980 (+ 575us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:39.783333 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.784161 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.785502 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.785923 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.786029 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.789690 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:39.789694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:39.789697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:39.789725 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:39.791985 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.792597 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.792601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.792604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.792727 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:39.793097 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:39.793099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:39.793101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:39.793388 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:39.793648 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:39.793873 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 01:00:39.794036 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:00:39.798661 8104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:00:39.806047 8104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:00:39.849668 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.820409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37672 (local address 127.5.108.131:46841)
0806 01:00:39.841019 (+ 20610us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:39.841026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:39.841764 (+ 738us) server_negotiation.cc:407] Connection header received
0806 01:00:39.842128 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:39.842138 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:39.842365 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:39.842552 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:39.843363 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.844633 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.847191 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.848570 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.848698 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.848939 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:39.849168 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 01:00:39.849343 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19375,"thread_start_us":19138,"threads_started":1}
I20260806 01:00:39.849810 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.818566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40480 (local address 127.5.108.129:42091)
0806 01:00:39.841204 (+ 22638us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:39.841215 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:39.841262 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:00:39.841379 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:39.841386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:39.841592 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:39.841954 (+ 362us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:39.842698 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.844188 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.846431 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.848182 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.848329 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.848547 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:39.849339 (+ 792us) server_negotiation.cc:299] Negotiation successful
0806 01:00:39.849500 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22461,"thread_start_us":22321,"threads_started":1}
I20260806 01:00:39.849709 8869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.818423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:46841 (local address 127.0.0.1:37672)
0806 01:00:39.841581 (+ 23158us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:39.841605 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:39.841899 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:39.842657 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:39.842662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:39.842687 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:39.843181 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.843190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.844819 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.844829 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.847028 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.847041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.848796 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.848802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.848969 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.849001 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:00:39.849108 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 01:00:39.849228 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23045,"mutex_wait_us":23,"thread_start_us":22896,"threads_started":1}
I20260806 01:00:39.850471 8866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.817868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:42091 (local address 127.0.0.1:40480)
0806 01:00:39.838910 (+ 21042us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:39.838954 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:39.839113 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:39.841888 (+ 2775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:39.841892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:39.841911 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:39.842468 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.842485 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.844355 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.844361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.846281 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.846293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.848433 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.848439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.848620 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.848681 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 01:00:39.848814 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 01:00:39.849029 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20762,"mutex_wait_us":53,"thread_start_us":20608,"threads_started":1}
I20260806 01:00:39.850488 8872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.819123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.5.108.130:39231)
0806 01:00:39.840717 (+ 21594us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:39.840726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:39.840758 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:00:39.840866 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:39.840875 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:39.841122 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:39.841423 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:39.842534 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.844185 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.847643 (+ 3458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.849016 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.849140 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.849340 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:39.850035 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 01:00:39.850210 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21430,"thread_start_us":21249,"threads_started":1}
I20260806 01:00:39.850567 8868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:39.818074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:39231 (local address 127.0.0.1:60372)
0806 01:00:39.840322 (+ 22248us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:39.840350 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:39.840482 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:39.841579 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:39.841586 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:39.841648 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:39.842256 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.842277 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.844396 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.844408 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.847463 (+ 3055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:39.847478 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:39.849481 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:39.849486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:39.849668 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:39.849713 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 01:00:39.849806 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 01:00:39.849914 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22077,"mutex_wait_us":91,"thread_start_us":21938,"threads_started":1}
I20260806 01:00:39.853703 8746 tablet_service.cc:1467] Processing CreateTablet for tablet bf98ac207a9b4b26bf862e7f9caed0a4 (DEFAULT_TABLE table=test-table [id=36dc9c98f2814560b515bbf03c287391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:39.854007 8547 tablet_service.cc:1467] Processing CreateTablet for tablet bf98ac207a9b4b26bf862e7f9caed0a4 (DEFAULT_TABLE table=test-table [id=36dc9c98f2814560b515bbf03c287391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:39.854009 8334 tablet_service.cc:1467] Processing CreateTablet for tablet bf98ac207a9b4b26bf862e7f9caed0a4 (DEFAULT_TABLE table=test-table [id=36dc9c98f2814560b515bbf03c287391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:39.855015 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf98ac207a9b4b26bf862e7f9caed0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:39.855283 8547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf98ac207a9b4b26bf862e7f9caed0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:39.855289 8334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf98ac207a9b4b26bf862e7f9caed0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:39.867761 8874 tablet_bootstrap.cc:492] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: Bootstrap starting.
I20260806 01:00:39.869890 8875 tablet_bootstrap.cc:492] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: Bootstrap starting.
I20260806 01:00:39.871351 8874 tablet_bootstrap.cc:654] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:39.872658 8874 log.cc:824] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:39.873443 8875 tablet_bootstrap.cc:654] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:39.874719 8875 log.cc:824] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:39.874070 8877 tablet_bootstrap.cc:492] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: Bootstrap starting.
I20260806 01:00:39.876134 8874 tablet_bootstrap.cc:492] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: No bootstrap required, opened a new log
I20260806 01:00:39.876412 8874 ts_tablet_manager.cc:1397] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 01:00:39.877436 8875 tablet_bootstrap.cc:492] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: No bootstrap required, opened a new log
I20260806 01:00:39.877749 8875 ts_tablet_manager.cc:1397] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 01:00:39.878702 8877 tablet_bootstrap.cc:654] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:39.879894 8877 log.cc:824] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:39.888509 8875 raft_consensus.cc:348] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:39.888911 8875 raft_consensus.cc:374] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:39.889062 8877 tablet_bootstrap.cc:492] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: No bootstrap required, opened a new log
I20260806 01:00:39.889035 8875 raft_consensus.cc:729] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 129b5aaf37054fc69f7936cf5b70f386, State: Initialized, Role: FOLLOWER
I20260806 01:00:39.888906 8874 raft_consensus.cc:348] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:39.889328 8877 ts_tablet_manager.cc:1397] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260806 01:00:39.889326 8874 raft_consensus.cc:374] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:39.889430 8874 raft_consensus.cc:729] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed2c8fcac404bf3a78b789a2b277978, State: Initialized, Role: FOLLOWER
I20260806 01:00:39.889839 8875 consensus_queue.cc:260] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:39.889990 8874 consensus_queue.cc:260] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:39.891619 8875 ts_tablet_manager.cc:1428] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260806 01:00:39.896304 8847 heartbeater.cc:503] Master 127.5.108.190:46751 was elected leader, sending a full tablet report...
I20260806 01:00:39.896560 8874 ts_tablet_manager.cc:1428] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: Time spent starting tablet: real 0.020s user 0.012s sys 0.007s
I20260806 01:00:39.900594 8877 raft_consensus.cc:348] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:39.901075 8877 raft_consensus.cc:374] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:39.901218 8877 raft_consensus.cc:729] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0f29970cc94da4b18a134fb024f348, State: Initialized, Role: FOLLOWER
I20260806 01:00:39.901777 8877 consensus_queue.cc:260] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:39.902693 8653 heartbeater.cc:503] Master 127.5.108.190:46751 was elected leader, sending a full tablet report...
I20260806 01:00:39.903079 8877 ts_tablet_manager.cc:1428] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260806 01:00:39.961014 8440 tablet.cc:2369] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:00:39.962888 8654 tablet.cc:2369] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:00:39.975638 8850 tablet.cc:2369] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:00:40.090538 8884 raft_consensus.cc:482] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:40.090864 8884 raft_consensus.cc:504] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.092656 8884 leader_election.cc:288] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129:42091), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:40.100015 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.093142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:48933 (local address 127.5.108.131:46841)
0806 01:00:40.093381 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.093390 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.094308 (+ 918us) server_negotiation.cc:407] Connection header received
0806 01:00:40.094471 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.094475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.094565 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.094680 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:40.095454 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.096269 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.098082 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.098955 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.099073 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.099172 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.099621 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.099763 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:00:40.100155 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.092977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:46841 (local address 127.5.108.130:48933)
0806 01:00:40.094169 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:40.094187 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:40.094339 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:40.094723 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:40.094726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:40.094842 (+ 116us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:40.095228 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.095241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.096404 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.096408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.097931 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.097949 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.099173 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.099179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.099425 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.099462 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:00:40.099551 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 01:00:40.099709 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"spinlock_wait_cycles":23424,"thread_start_us":665,"threads_started":1}
I20260806 01:00:40.100164 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.093073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:56347 (local address 127.5.108.129:42091)
0806 01:00:40.093342 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.093351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.094162 (+ 811us) server_negotiation.cc:407] Connection header received
0806 01:00:40.094343 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.094348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.094444 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.094520 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:40.095143 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.095943 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.098173 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.099028 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.099193 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.099273 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.099793 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.099949 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 01:00:40.100586 8887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.092930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:42091 (local address 127.5.108.130:56347)
0806 01:00:40.094045 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:40.094061 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:40.094194 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:40.094567 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:40.094572 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:40.094598 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:40.094998 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.095008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.096073 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.096078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.097630 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.097642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.099258 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.099263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.099509 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.099554 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 01:00:40.099740 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 01:00:40.099851 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"thread_start_us":786,"threads_started":1}
I20260806 01:00:40.101094 8777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf98ac207a9b4b26bf862e7f9caed0a4" candidate_uuid: "bd0f29970cc94da4b18a134fb024f348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978" is_pre_election: true
I20260806 01:00:40.101589 8777 raft_consensus.cc:2454] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd0f29970cc94da4b18a134fb024f348 in term 0.
I20260806 01:00:40.101534 8368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf98ac207a9b4b26bf862e7f9caed0a4" candidate_uuid: "bd0f29970cc94da4b18a134fb024f348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "129b5aaf37054fc69f7936cf5b70f386" is_pre_election: true
I20260806 01:00:40.101999 8368 raft_consensus.cc:2454] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd0f29970cc94da4b18a134fb024f348 in term 0.
I20260806 01:00:40.102283 8475 leader_election.cc:302] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ed2c8fcac404bf3a78b789a2b277978, bd0f29970cc94da4b18a134fb024f348; no voters:
I20260806 01:00:40.102653 8884 raft_consensus.cc:2781] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:00:40.102818 8884 raft_consensus.cc:482] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:00:40.102965 8884 raft_consensus.cc:3037] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:40.184453 8884 raft_consensus.cc:504] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.185388 8884 leader_election.cc:288] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [CANDIDATE]: Term 1 election: Requested vote from peers 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129:42091), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:40.185911 8777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf98ac207a9b4b26bf862e7f9caed0a4" candidate_uuid: "bd0f29970cc94da4b18a134fb024f348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978"
I20260806 01:00:40.185935 8368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf98ac207a9b4b26bf862e7f9caed0a4" candidate_uuid: "bd0f29970cc94da4b18a134fb024f348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "129b5aaf37054fc69f7936cf5b70f386"
I20260806 01:00:40.186151 8777 raft_consensus.cc:3037] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:40.186177 8368 raft_consensus.cc:3037] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:40.188485 8777 raft_consensus.cc:2454] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd0f29970cc94da4b18a134fb024f348 in term 1.
I20260806 01:00:40.188979 8475 leader_election.cc:302] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ed2c8fcac404bf3a78b789a2b277978, bd0f29970cc94da4b18a134fb024f348; no voters:
I20260806 01:00:40.189282 8884 raft_consensus.cc:2781] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:40.189848 8884 raft_consensus.cc:686] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 1 LEADER]: Becoming Leader. State: Replica: bd0f29970cc94da4b18a134fb024f348, State: Running, Role: LEADER
I20260806 01:00:40.190382 8884 consensus_queue.cc:237] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.195202 8103 catalog_manager.cc:5495] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd0f29970cc94da4b18a134fb024f348 (127.5.108.130). New cstate: current_term: 1 leader_uuid: "bd0f29970cc94da4b18a134fb024f348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } health_report { overall_health: UNKNOWN } } }
I20260806 01:00:40.287747 8368 raft_consensus.cc:2454] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd0f29970cc94da4b18a134fb024f348 in term 1.
Aug 06 01:00:40 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:40.311700 8898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.297481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46751 (local address 127.0.0.1:40134)
0806 01:00:40.298343 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:40.298360 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:40.298474 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:40.299277 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:40.299281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:40.300059 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:40.300341 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.300350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.301455 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.301458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.302394 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.302405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.303094 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.303098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.303233 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.304043 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:40.304060 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:40.306483 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:40.309434 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:40.309445 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:40.309450 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:40.309885 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:40.310192 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:40.310195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:40.310197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:40.310340 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:40.310812 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:40.310818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:40.310962 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 01:00:40.311114 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 01:00:40.311328 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":402,"threads_started":1}
I20260806 01:00:40.311858 8900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.297508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40134 (local address 127.5.108.190:46751)
0806 01:00:40.298426 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.298430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.298452 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:00:40.299051 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.299055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.299114 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.299213 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:40.300491 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.301343 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.302509 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.302924 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.303021 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.306685 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:40.306690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:40.306693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:40.306722 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:40.309311 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:40.309986 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:40.309989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:40.309992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:40.310102 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:40.310434 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:40.310437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:40.310439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:40.310700 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:40.310808 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.311195 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.311540 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":777,"thread_start_us":586,"threads_started":1}
I20260806 01:00:40.316733 8103 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:40134:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 01:00:40.317870 8103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:00:40.324311 8334 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa4e1e3738e4ed5ba182329fbf81c96 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=055e9d2070024859b3dedb4e2bbe7c29]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:00:40.324733 8746 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa4e1e3738e4ed5ba182329fbf81c96 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=055e9d2070024859b3dedb4e2bbe7c29]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:00:40.324693 8547 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa4e1e3738e4ed5ba182329fbf81c96 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=055e9d2070024859b3dedb4e2bbe7c29]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:00:40.325408 8334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa4e1e3738e4ed5ba182329fbf81c96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:40.325506 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa4e1e3738e4ed5ba182329fbf81c96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:40.325596 8547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa4e1e3738e4ed5ba182329fbf81c96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:40.331192 8877 tablet_bootstrap.cc:492] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: Bootstrap starting.
I20260806 01:00:40.332682 8874 tablet_bootstrap.cc:492] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: Bootstrap starting.
I20260806 01:00:40.333200 8875 tablet_bootstrap.cc:492] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: Bootstrap starting.
I20260806 01:00:40.333355 8877 tablet_bootstrap.cc:654] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:40.334498 8874 tablet_bootstrap.cc:654] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:40.334645 8875 tablet_bootstrap.cc:654] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:40.336040 8877 tablet_bootstrap.cc:492] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: No bootstrap required, opened a new log
I20260806 01:00:40.336223 8877 ts_tablet_manager.cc:1397] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260806 01:00:40.336810 8874 tablet_bootstrap.cc:492] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: No bootstrap required, opened a new log
I20260806 01:00:40.336969 8874 ts_tablet_manager.cc:1397] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260806 01:00:40.336920 8877 raft_consensus.cc:348] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.337164 8877 raft_consensus.cc:374] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:40.337296 8877 raft_consensus.cc:729] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0f29970cc94da4b18a134fb024f348, State: Initialized, Role: FOLLOWER
I20260806 01:00:40.337599 8877 consensus_queue.cc:260] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.337827 8875 tablet_bootstrap.cc:492] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: No bootstrap required, opened a new log
I20260806 01:00:40.337957 8875 ts_tablet_manager.cc:1397] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 01:00:40.338266 8877 ts_tablet_manager.cc:1428] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 01:00:40.338102 8884 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.338442 8884 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:40.337791 8874 raft_consensus.cc:348] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.338840 8874 raft_consensus.cc:374] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:40.338747 8875 raft_consensus.cc:348] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.338949 8874 raft_consensus.cc:729] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed2c8fcac404bf3a78b789a2b277978, State: Initialized, Role: FOLLOWER
I20260806 01:00:40.339002 8875 raft_consensus.cc:374] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:40.339123 8875 raft_consensus.cc:729] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 129b5aaf37054fc69f7936cf5b70f386, State: Initialized, Role: FOLLOWER
I20260806 01:00:40.339177 8874 consensus_queue.cc:260] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.339363 8875 consensus_queue.cc:260] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.339833 8874 ts_tablet_manager.cc:1428] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 01:00:40.339982 8875 ts_tablet_manager.cc:1428] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 01:00:40.340273 8882 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.340648 8882 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:40.340972 8883 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.341264 8883 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:40.430025 8882 raft_consensus.cc:482] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:40.430361 8882 raft_consensus.cc:504] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.432044 8882 leader_election.cc:288] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:40.441588 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.432588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:47593 (local address 127.5.108.131:46841)
0806 01:00:40.432840 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.432851 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.433491 (+ 640us) server_negotiation.cc:407] Connection header received
0806 01:00:40.433845 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.433851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.433948 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.434060 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:40.434789 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.435981 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.439021 (+ 3040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.440306 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.440469 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.440866 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.441101 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.441226 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:00:40.441608 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.432482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:43247 (local address 127.5.108.130:39231)
0806 01:00:40.433645 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.433652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.434050 (+ 398us) server_negotiation.cc:407] Connection header received
0806 01:00:40.434261 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.434265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.434321 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.434399 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:00:40.435109 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.436245 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.439012 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.440316 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.440477 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.440544 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.441078 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.441225 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":850,"threads_started":1}
I20260806 01:00:40.441979 8906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.432302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:46841 (local address 127.5.108.129:47593)
0806 01:00:40.433371 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:40.433395 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:40.433644 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:40.434095 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:40.434100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:40.434125 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:40.434567 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.434587 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.436133 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.436138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.438849 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.438859 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.440646 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.440651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.440940 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.440983 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 01:00:40.441065 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 01:00:40.441173 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":129,"thread_start_us":723,"threads_started":1}
I20260806 01:00:40.442004 8907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.432408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:39231 (local address 127.5.108.129:43247)
0806 01:00:40.433937 (+ 1529us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:40.433951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:40.434099 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:40.434473 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:40.434481 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:40.434520 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:00:40.434943 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.434953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.436398 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.436404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.438840 (+ 2436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.438858 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.440622 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.440634 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.440875 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.440914 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 01:00:40.441022 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 01:00:40.441161 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1409,"mutex_wait_us":109,"thread_start_us":1212,"threads_started":1}
I20260806 01:00:40.442723 8777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa4e1e3738e4ed5ba182329fbf81c96" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978" is_pre_election: true
I20260806 01:00:40.443032 8777 raft_consensus.cc:2454] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 0.
I20260806 01:00:40.443020 8576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa4e1e3738e4ed5ba182329fbf81c96" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0f29970cc94da4b18a134fb024f348" is_pre_election: true
I20260806 01:00:40.443481 8576 raft_consensus.cc:2454] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 0.
I20260806 01:00:40.443578 8242 leader_election.cc:302] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 129b5aaf37054fc69f7936cf5b70f386, 5ed2c8fcac404bf3a78b789a2b277978; no voters:
I20260806 01:00:40.444032 8882 raft_consensus.cc:2781] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:00:40.444150 8882 raft_consensus.cc:482] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:00:40.444274 8882 raft_consensus.cc:3037] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:40.492755 8882 raft_consensus.cc:504] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.493800 8882 leader_election.cc:288] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 election: Requested vote from peers bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:40.494390 8777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa4e1e3738e4ed5ba182329fbf81c96" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978"
I20260806 01:00:40.494393 8576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa4e1e3738e4ed5ba182329fbf81c96" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0f29970cc94da4b18a134fb024f348"
I20260806 01:00:40.494644 8777 raft_consensus.cc:3037] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:40.494673 8576 raft_consensus.cc:3037] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:40.497141 8777 raft_consensus.cc:2454] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 1.
I20260806 01:00:40.497648 8242 leader_election.cc:302] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 129b5aaf37054fc69f7936cf5b70f386, 5ed2c8fcac404bf3a78b789a2b277978; no voters:
I20260806 01:00:40.497977 8882 raft_consensus.cc:2781] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:40.498562 8882 raft_consensus.cc:686] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 LEADER]: Becoming Leader. State: Replica: 129b5aaf37054fc69f7936cf5b70f386, State: Running, Role: LEADER
I20260806 01:00:40.498710 8576 raft_consensus.cc:2454] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 1.
I20260806 01:00:40.499224 8882 consensus_queue.cc:237] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.503181 8909 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: TxnStatusTablet state changed. Reason: New leader 129b5aaf37054fc69f7936cf5b70f386. Latest consensus state: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.503614 8909 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:00:40.504352 8912 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 01:00:40.504678 8912 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 01:00:40.505002 8100 catalog_manager.cc:5495] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 reported cstate change: term changed from 0 to 1, leader changed from <none> to 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129). New cstate: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } health_report { overall_health: UNKNOWN } } }
I20260806 01:00:40.536170 8898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.530008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:42091 (local address 127.0.0.1:38860)
0806 01:00:40.530251 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:40.530271 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:40.530421 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:40.530919 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:40.530924 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:40.530937 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:00:40.531141 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.531152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.532439 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.532447 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.533496 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.533510 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.534312 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.534319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.534489 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.534565 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.535512 (+ 947us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.535537 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 01:00:40.535663 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 01:00:40.535822 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 01:00:40.536182 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.530095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38860 (local address 127.5.108.129:42091)
0806 01:00:40.530253 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.530260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.530354 (+ 94us) server_negotiation.cc:407] Connection header received
0806 01:00:40.530655 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.530659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.530740 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.530856 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:40.531327 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.532280 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.533666 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.534117 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.534297 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.534813 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.535343 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.535552 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.535691 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.535784 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 01:00:40.545049 8576 raft_consensus.cc:1261] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Refusing update from remote peer 129b5aaf37054fc69f7936cf5b70f386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:40.545357 8777 raft_consensus.cc:1261] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Refusing update from remote peer 129b5aaf37054fc69f7936cf5b70f386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:40.546065 8909 consensus_queue.cc:1035] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:40.546453 8882 consensus_queue.cc:1035] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:40.554364 8883 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: TxnStatusTablet state changed. Reason: New leader 129b5aaf37054fc69f7936cf5b70f386. Latest consensus state: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.554683 8883 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:40.558265 8918 mvcc.cc:205] Tried to move back new op lower bound from 7315366054061563904 to 7315366053889163264. Current Snapshot: MvccSnapshot[applied={T|T < 7315366054061563904}]
I20260806 01:00:40.562137 8882 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.562556 8882 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:00:40.566457 8884 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: TxnStatusTablet state changed. Reason: New leader 129b5aaf37054fc69f7936cf5b70f386. Latest consensus state: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.566876 8884 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:40.569011 8909 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.569880 8909 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 129b5aaf37054fc69f7936cf5b70f386: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:00:40.573001 8883 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.573254 8883 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P 5ed2c8fcac404bf3a78b789a2b277978: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:40.576004 8884 tablet_replica.cc:439] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } } }
I20260806 01:00:40.576332 8884 tablet_replica.cc:442] T 7fa4e1e3738e4ed5ba182329fbf81c96 P bd0f29970cc94da4b18a134fb024f348: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:00:40.584499 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.576645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38008 (local address 127.5.108.130:39231)
0806 01:00:40.576831 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.576838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.577759 (+ 921us) server_negotiation.cc:407] Connection header received
0806 01:00:40.578213 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.578221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.578337 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.578463 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:40.579322 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.580525 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.581937 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.582521 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.582638 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.583186 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.583855 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.584001 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.584088 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.584170 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:00:40.603055 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.596184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:41031 (local address 127.5.108.129:42091)
0806 01:00:40.596405 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.596412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.597069 (+ 657us) server_negotiation.cc:407] Connection header received
0806 01:00:40.597238 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.597247 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.597372 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.597487 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:40.598081 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.599040 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.601254 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.601735 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.601878 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.602383 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.602491 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.602568 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.602730 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.602801 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:00:40.603163 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.596124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:42091 (local address 127.5.108.130:41031)
0806 01:00:40.596923 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:40.596949 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:40.597099 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:40.597517 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:40.597522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:40.597535 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:00:40.597886 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.597898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.599192 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.599197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.601100 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.601116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.601931 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.601938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.602111 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.602179 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.602598 (+ 419us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.602609 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:00:40.602696 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 01:00:40.602832 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":453,"threads_started":1}
I20260806 01:00:40.613606 8777 raft_consensus.cc:1261] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Refusing update from remote peer bd0f29970cc94da4b18a134fb024f348: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:40.613927 8368 raft_consensus.cc:1261] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Refusing update from remote peer bd0f29970cc94da4b18a134fb024f348: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:40.614377 8888 consensus_queue.cc:1035] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:40.614727 8884 consensus_queue.cc:1035] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:40.620438 8921 mvcc.cc:205] Tried to move back new op lower bound from 7315366054346027008 to 7315366052623736832. Current Snapshot: MvccSnapshot[applied={T|T < 7315366054346027008}]
I20260806 01:00:40.621102 8919 mvcc.cc:205] Tried to move back new op lower bound from 7315366054346027008 to 7315366052623736832. Current Snapshot: MvccSnapshot[applied={T|T < 7315366054346027008}]
I20260806 01:00:40.653596 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.647200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:54085 (local address 127.5.108.130:39231)
0806 01:00:40.647425 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.647431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.647687 (+ 256us) server_negotiation.cc:407] Connection header received
0806 01:00:40.647931 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.647935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.648005 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.648106 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:40.648850 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.650032 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.651685 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.652171 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.652321 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.652792 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.652908 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.653008 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.653190 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.653278 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 01:00:40.653728 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.646719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:39231 (local address 127.5.108.129:54085)
0806 01:00:40.647578 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:40.647593 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:40.647752 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:40.648286 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:40.648290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:40.648300 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:00:40.648661 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.648669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.650172 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.650178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.651536 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.651549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.652390 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.652398 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.652535 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.652588 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.653028 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.653043 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 01:00:40.653164 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 01:00:40.653305 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":403,"threads_started":1}
I20260806 01:00:40.705585 8101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:40.706050 8101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:40.710346 8101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac: Sending DeleteTablet for 3 replicas of tablet bf98ac207a9b4b26bf862e7f9caed0a4
I20260806 01:00:40.711911 8547 tablet_service.cc:1514] Processing DeleteTablet for tablet bf98ac207a9b4b26bf862e7f9caed0a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:40 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:60372
I20260806 01:00:40.713806 8746 tablet_service.cc:1514] Processing DeleteTablet for tablet bf98ac207a9b4b26bf862e7f9caed0a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:40 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:37672
I20260806 01:00:40.714102 8334 tablet_service.cc:1514] Processing DeleteTablet for tablet bf98ac207a9b4b26bf862e7f9caed0a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:40 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:40480
I20260806 01:00:40.716192 8101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:00:40.716864 8101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:00:40.726526 8548 tablet_service.cc:1467] Processing CreateTablet for tablet bfa890780acb4a13a8cf74d305268066 (DEFAULT_TABLE table=test-table [id=ec803f36ec9b447b85180756c3cec6ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:40.726670 8332 tablet_service.cc:1467] Processing CreateTablet for tablet bfa890780acb4a13a8cf74d305268066 (DEFAULT_TABLE table=test-table [id=ec803f36ec9b447b85180756c3cec6ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:40.726965 8745 tablet_service.cc:1467] Processing CreateTablet for tablet bfa890780acb4a13a8cf74d305268066 (DEFAULT_TABLE table=test-table [id=ec803f36ec9b447b85180756c3cec6ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:40.727190 8548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa890780acb4a13a8cf74d305268066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:40.727305 8332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa890780acb4a13a8cf74d305268066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:40.727555 8745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa890780acb4a13a8cf74d305268066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:40.734261 8877 tablet_bootstrap.cc:492] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: Bootstrap starting.
I20260806 01:00:40.735600 8875 tablet_bootstrap.cc:492] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: Bootstrap starting.
I20260806 01:00:40.736152 8877 tablet_bootstrap.cc:654] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:40.737773 8875 tablet_bootstrap.cc:654] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:40.746980 8948 tablet_replica.cc:330] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: stopping tablet replica
I20260806 01:00:40.747475 8948 raft_consensus.cc:2229] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:00:40.748188 8948 raft_consensus.cc:2258] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:40.749027 8952 tablet_replica.cc:330] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: stopping tablet replica
I20260806 01:00:40.749457 8952 raft_consensus.cc:2229] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:40.749778 8952 raft_consensus.cc:2258] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:40.751799 8948 ts_tablet_manager.cc:1905] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:40.757336 8952 ts_tablet_manager.cc:1905] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:40.758162 8875 tablet_bootstrap.cc:492] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: No bootstrap required, opened a new log
I20260806 01:00:40.758363 8875 ts_tablet_manager.cc:1397] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.008s
I20260806 01:00:40.759231 8875 raft_consensus.cc:348] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.759539 8875 raft_consensus.cc:374] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:40.759631 8875 raft_consensus.cc:729] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 129b5aaf37054fc69f7936cf5b70f386, State: Initialized, Role: FOLLOWER
I20260806 01:00:40.759917 8875 consensus_queue.cc:260] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.760407 8874 tablet_bootstrap.cc:492] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: Bootstrap starting.
I20260806 01:00:40.762344 8874 tablet_bootstrap.cc:654] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:40.765870 8875 ts_tablet_manager.cc:1428] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260806 01:00:40.768091 8950 tablet_replica.cc:330] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: stopping tablet replica
I20260806 01:00:40.768499 8950 raft_consensus.cc:2229] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:40.768882 8950 raft_consensus.cc:2258] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:40.770571 8950 ts_tablet_manager.cc:1905] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:40.773257 8877 tablet_bootstrap.cc:492] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: No bootstrap required, opened a new log
I20260806 01:00:40.773514 8877 ts_tablet_manager.cc:1397] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: Time spent bootstrapping tablet: real 0.039s user 0.009s sys 0.001s
I20260806 01:00:40.774788 8877 raft_consensus.cc:348] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.775342 8877 raft_consensus.cc:374] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:40.775527 8877 raft_consensus.cc:729] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0f29970cc94da4b18a134fb024f348, State: Initialized, Role: FOLLOWER
I20260806 01:00:40.775987 8877 consensus_queue.cc:260] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.776794 8877 ts_tablet_manager.cc:1428] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 01:00:40.776850 8874 tablet_bootstrap.cc:492] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: No bootstrap required, opened a new log
I20260806 01:00:40.777042 8874 ts_tablet_manager.cc:1397] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.005s
I20260806 01:00:40.778090 8874 raft_consensus.cc:348] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.778403 8874 raft_consensus.cc:374] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:40.778509 8874 raft_consensus.cc:729] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed2c8fcac404bf3a78b789a2b277978, State: Initialized, Role: FOLLOWER
I20260806 01:00:40.778829 8874 consensus_queue.cc:260] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.779667 8874 ts_tablet_manager.cc:1428] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 01:00:40.782208 8935 raft_consensus.cc:482] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:40.782382 8935 raft_consensus.cc:504] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.783249 8935 leader_election.cc:288] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:40.783653 8576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa890780acb4a13a8cf74d305268066" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0f29970cc94da4b18a134fb024f348" is_pre_election: true
I20260806 01:00:40.783845 8576 raft_consensus.cc:2454] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 0.
I20260806 01:00:40.783950 8776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa890780acb4a13a8cf74d305268066" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978" is_pre_election: true
I20260806 01:00:40.784240 8776 raft_consensus.cc:2454] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 0.
I20260806 01:00:40.784268 8244 leader_election.cc:302] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 129b5aaf37054fc69f7936cf5b70f386, bd0f29970cc94da4b18a134fb024f348; no voters:
I20260806 01:00:40.785125 8935 raft_consensus.cc:2781] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:00:40.785257 8935 raft_consensus.cc:482] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:00:40.785343 8935 raft_consensus.cc:3037] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:40.805960 8948 ts_tablet_manager.cc:1918] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:00:40.806398 8948 log.cc:1196] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/bf98ac207a9b4b26bf862e7f9caed0a4
I20260806 01:00:40.807011 8948 ts_tablet_manager.cc:1939] T bf98ac207a9b4b26bf862e7f9caed0a4 P bd0f29970cc94da4b18a134fb024f348: Deleting consensus metadata
I20260806 01:00:40.807214 8935 raft_consensus.cc:504] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.807858 8935 leader_election.cc:288] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 election: Requested vote from peers bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:40.808801 8576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa890780acb4a13a8cf74d305268066" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0f29970cc94da4b18a134fb024f348"
I20260806 01:00:40.809059 8576 raft_consensus.cc:3037] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:40.812366 8776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa890780acb4a13a8cf74d305268066" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978"
I20260806 01:00:40.812613 8776 raft_consensus.cc:3037] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:40.867421 8088 catalog_manager.cc:4841] TS bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231): tablet bf98ac207a9b4b26bf862e7f9caed0a4 (table test-table [id=36dc9c98f2814560b515bbf03c287391]) successfully deleted
I20260806 01:00:40.895483 8950 ts_tablet_manager.cc:1918] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:00:40.895484 8952 ts_tablet_manager.cc:1918] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:00:40.895704 8950 log.cc:1196] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/bf98ac207a9b4b26bf862e7f9caed0a4
I20260806 01:00:40.895709 8952 log.cc:1196] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/bf98ac207a9b4b26bf862e7f9caed0a4
I20260806 01:00:40.896312 8950 ts_tablet_manager.cc:1939] T bf98ac207a9b4b26bf862e7f9caed0a4 P 5ed2c8fcac404bf3a78b789a2b277978: Deleting consensus metadata
I20260806 01:00:40.896308 8952 ts_tablet_manager.cc:1939] T bf98ac207a9b4b26bf862e7f9caed0a4 P 129b5aaf37054fc69f7936cf5b70f386: Deleting consensus metadata
I20260806 01:00:40.896852 8776 raft_consensus.cc:2454] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 1.
I20260806 01:00:40.897310 8576 raft_consensus.cc:2454] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 1.
I20260806 01:00:40.898550 8242 leader_election.cc:302] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 129b5aaf37054fc69f7936cf5b70f386, 5ed2c8fcac404bf3a78b789a2b277978; no voters:
I20260806 01:00:40.898949 8935 raft_consensus.cc:2781] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:40.899089 8935 raft_consensus.cc:686] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 LEADER]: Becoming Leader. State: Replica: 129b5aaf37054fc69f7936cf5b70f386, State: Running, Role: LEADER
I20260806 01:00:40.900533 8935 consensus_queue.cc:237] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:40.906255 8087 catalog_manager.cc:4841] TS 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841): tablet bf98ac207a9b4b26bf862e7f9caed0a4 (table test-table [id=36dc9c98f2814560b515bbf03c287391]) successfully deleted
I20260806 01:00:40.909039 8085 catalog_manager.cc:4841] TS 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129:42091): tablet bf98ac207a9b4b26bf862e7f9caed0a4 (table test-table [id=36dc9c98f2814560b515bbf03c287391]) successfully deleted
I20260806 01:00:40.911481 8101 catalog_manager.cc:5495] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 reported cstate change: term changed from 0 to 1, leader changed from <none> to 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129). New cstate: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } health_report { overall_health: UNKNOWN } } }
I20260806 01:00:40.968540 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.962006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.5.108.129:42091)
0806 01:00:40.962193 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.962200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.962257 (+ 57us) server_negotiation.cc:407] Connection header received
0806 01:00:40.962478 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.962483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.962571 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.962677 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:40.963093 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.964373 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.966231 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.966943 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.967082 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.967701 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.967849 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.967954 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.968133 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.968219 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:00:40.985870 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.978468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:57601 (local address 127.5.108.129:42091)
0806 01:00:40.978699 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:40.978707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:40.978895 (+ 188us) server_negotiation.cc:407] Connection header received
0806 01:00:40.979097 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:40.979103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:40.979194 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:40.979312 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:40.979743 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.980945 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.983610 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.984317 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.984533 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.984957 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.985128 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.985229 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:40.985407 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 01:00:40.985503 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:00:40.986446 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:40.978504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:42091 (local address 127.5.108.129:57601)
0806 01:00:40.978795 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:40.978814 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:40.978956 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:40.979333 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:40.979338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:40.979355 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:00:40.979579 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.979591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.981121 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.981126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.982874 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:40.982889 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:40.984524 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:40.984530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:40.984706 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:40.984758 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:00:40.985249 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:00:40.985261 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 01:00:40.985357 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 01:00:40.985914 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":1}
I20260806 01:00:40.998324 8776 raft_consensus.cc:1261] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Refusing update from remote peer 129b5aaf37054fc69f7936cf5b70f386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:40.998378 8576 raft_consensus.cc:1261] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Refusing update from remote peer 129b5aaf37054fc69f7936cf5b70f386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:40.999050 8882 consensus_queue.cc:1035] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:40.999457 8909 consensus_queue.cc:1035] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:00:41.055194 8918 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 01:00:42.662408 8101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:42.662667 8101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:42.666886 8101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac: Sending DeleteTablet for 3 replicas of tablet bfa890780acb4a13a8cf74d305268066
I20260806 01:00:42.667676 8547 tablet_service.cc:1514] Processing DeleteTablet for tablet bfa890780acb4a13a8cf74d305268066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:42 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:60372
I20260806 01:00:42.667676 8334 tablet_service.cc:1514] Processing DeleteTablet for tablet bfa890780acb4a13a8cf74d305268066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:42 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:40480
I20260806 01:00:42.667788 8746 tablet_service.cc:1514] Processing DeleteTablet for tablet bfa890780acb4a13a8cf74d305268066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:42 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:37672
I20260806 01:00:42.680107 8992 tablet_replica.cc:330] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: stopping tablet replica
I20260806 01:00:42.680337 8992 raft_consensus.cc:2229] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:00:42.680393 8995 tablet_replica.cc:330] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: stopping tablet replica
I20260806 01:00:42.680682 8995 raft_consensus.cc:2229] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:42.680938 8992 raft_consensus.cc:2258] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:42.681007 8995 raft_consensus.cc:2258] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:42.682372 8995 ts_tablet_manager.cc:1905] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:42.683871 8992 ts_tablet_manager.cc:1905] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:42.685029 8993 tablet_replica.cc:330] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: stopping tablet replica
I20260806 01:00:42.685246 8993 raft_consensus.cc:2229] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:42.685470 8993 raft_consensus.cc:2258] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:42.686549 8993 ts_tablet_manager.cc:1905] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:42.691488 8995 ts_tablet_manager.cc:1918] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:00:42.691637 8995 log.cc:1196] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/bfa890780acb4a13a8cf74d305268066
I20260806 01:00:42.692106 8995 ts_tablet_manager.cc:1939] T bfa890780acb4a13a8cf74d305268066 P 5ed2c8fcac404bf3a78b789a2b277978: Deleting consensus metadata
I20260806 01:00:42.692777 8992 ts_tablet_manager.cc:1918] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:00:42.692998 8992 log.cc:1196] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/bfa890780acb4a13a8cf74d305268066
I20260806 01:00:42.693231 8087 catalog_manager.cc:4841] TS 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841): tablet bfa890780acb4a13a8cf74d305268066 (table test-table [id=ec803f36ec9b447b85180756c3cec6ee]) successfully deleted
I20260806 01:00:42.693681 8992 ts_tablet_manager.cc:1939] T bfa890780acb4a13a8cf74d305268066 P 129b5aaf37054fc69f7936cf5b70f386: Deleting consensus metadata
I20260806 01:00:42.694139 8993 ts_tablet_manager.cc:1918] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:00:42.694324 8993 log.cc:1196] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/bfa890780acb4a13a8cf74d305268066
I20260806 01:00:42.694785 8993 ts_tablet_manager.cc:1939] T bfa890780acb4a13a8cf74d305268066 P bd0f29970cc94da4b18a134fb024f348: Deleting consensus metadata
I20260806 01:00:42.694754 8085 catalog_manager.cc:4841] TS 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129:42091): tablet bfa890780acb4a13a8cf74d305268066 (table test-table [id=ec803f36ec9b447b85180756c3cec6ee]) successfully deleted
I20260806 01:00:42.695837 8088 catalog_manager.cc:4841] TS bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231): tablet bfa890780acb4a13a8cf74d305268066 (table test-table [id=ec803f36ec9b447b85180756c3cec6ee]) successfully deleted
Aug 06 01:00:42 dist-test-slave-x4qh krb5kdc[8040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:00:42 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:42.741604 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:42.720848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.5.108.190:46751)
0806 01:00:42.723420 (+ 2572us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:42.723426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:42.723447 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:00:42.723504 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:42.723509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:42.723584 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:42.723691 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:42.726047 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:42.727183 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:42.728883 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:42.729434 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:42.729555 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:42.734331 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:42.734338 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:42.734343 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:42.734378 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:42.738883 (+ 4505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:42.739598 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:42.739604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:42.739608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:42.739743 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:42.740071 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:42.740075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:42.740079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:42.740490 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:42.740753 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:42.740933 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 01:00:42.741154 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2422,"thread_start_us":2265,"threads_started":1}
I20260806 01:00:42.745599 8101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:00:42.746299 8101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:00:42.752885 8334 tablet_service.cc:1467] Processing CreateTablet for tablet 078e106eeb9642eb9c5f5dcd6fb279fb (DEFAULT_TABLE table=test-table [id=6c97883cba1f499fb62717442f17dcee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:42.753340 8334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 078e106eeb9642eb9c5f5dcd6fb279fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:42.753473 8746 tablet_service.cc:1467] Processing CreateTablet for tablet 078e106eeb9642eb9c5f5dcd6fb279fb (DEFAULT_TABLE table=test-table [id=6c97883cba1f499fb62717442f17dcee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:42.753968 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 078e106eeb9642eb9c5f5dcd6fb279fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:42.759311 8547 tablet_service.cc:1467] Processing CreateTablet for tablet 078e106eeb9642eb9c5f5dcd6fb279fb (DEFAULT_TABLE table=test-table [id=6c97883cba1f499fb62717442f17dcee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:42.759819 8547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 078e106eeb9642eb9c5f5dcd6fb279fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:42.760622 9008 tablet_bootstrap.cc:492] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: Bootstrap starting.
I20260806 01:00:42.763576 9008 tablet_bootstrap.cc:654] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:42.763828 9009 tablet_bootstrap.cc:492] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: Bootstrap starting.
I20260806 01:00:42.766096 9008 tablet_bootstrap.cc:492] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: No bootstrap required, opened a new log
I20260806 01:00:42.766260 9008 ts_tablet_manager.cc:1397] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260806 01:00:42.766618 9009 tablet_bootstrap.cc:654] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:42.767609 9008 raft_consensus.cc:348] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:42.767912 9008 raft_consensus.cc:374] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:42.768091 9008 raft_consensus.cc:729] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 129b5aaf37054fc69f7936cf5b70f386, State: Initialized, Role: FOLLOWER
I20260806 01:00:42.768424 9008 consensus_queue.cc:260] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:42.769155 9008 ts_tablet_manager.cc:1428] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 01:00:42.769876 9014 tablet_bootstrap.cc:492] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: Bootstrap starting.
I20260806 01:00:42.771750 9014 tablet_bootstrap.cc:654] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:42.773306 9009 tablet_bootstrap.cc:492] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: No bootstrap required, opened a new log
I20260806 01:00:42.773461 9009 ts_tablet_manager.cc:1397] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260806 01:00:42.774286 9009 raft_consensus.cc:348] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:42.774412 9014 tablet_bootstrap.cc:492] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: No bootstrap required, opened a new log
I20260806 01:00:42.774480 9009 raft_consensus.cc:374] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:42.774564 9014 ts_tablet_manager.cc:1397] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 01:00:42.774595 9009 raft_consensus.cc:729] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed2c8fcac404bf3a78b789a2b277978, State: Initialized, Role: FOLLOWER
I20260806 01:00:42.774822 9009 consensus_queue.cc:260] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:42.775424 9014 raft_consensus.cc:348] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:42.775770 9009 ts_tablet_manager.cc:1428] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 01:00:42.775825 9014 raft_consensus.cc:374] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:42.775971 9014 raft_consensus.cc:729] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0f29970cc94da4b18a134fb024f348, State: Initialized, Role: FOLLOWER
I20260806 01:00:42.776242 9014 consensus_queue.cc:260] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:42.777185 9014 ts_tablet_manager.cc:1428] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 01:00:42.961304 8882 raft_consensus.cc:482] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:42.961566 8882 raft_consensus.cc:504] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:42.962265 8882 leader_election.cc:288] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:42.962841 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078e106eeb9642eb9c5f5dcd6fb279fb" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0f29970cc94da4b18a134fb024f348" is_pre_election: true
I20260806 01:00:42.962913 8777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078e106eeb9642eb9c5f5dcd6fb279fb" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978" is_pre_election: true
I20260806 01:00:42.963114 8572 raft_consensus.cc:2454] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 0.
I20260806 01:00:42.963212 8777 raft_consensus.cc:2454] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 0.
I20260806 01:00:42.963676 8244 leader_election.cc:302] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 129b5aaf37054fc69f7936cf5b70f386, bd0f29970cc94da4b18a134fb024f348; no voters:
I20260806 01:00:42.963989 8882 raft_consensus.cc:2781] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:00:42.964097 8882 raft_consensus.cc:482] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:00:42.964284 8882 raft_consensus.cc:3037] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:42.966547 8882 raft_consensus.cc:504] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:42.967132 8882 leader_election.cc:288] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 election: Requested vote from peers bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:42.967523 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078e106eeb9642eb9c5f5dcd6fb279fb" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0f29970cc94da4b18a134fb024f348"
I20260806 01:00:42.967689 8777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078e106eeb9642eb9c5f5dcd6fb279fb" candidate_uuid: "129b5aaf37054fc69f7936cf5b70f386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978"
I20260806 01:00:42.967787 8572 raft_consensus.cc:3037] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:42.967917 8777 raft_consensus.cc:3037] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:42.969764 8572 raft_consensus.cc:2454] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 1.
I20260806 01:00:42.969900 8777 raft_consensus.cc:2454] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 129b5aaf37054fc69f7936cf5b70f386 in term 1.
I20260806 01:00:42.970196 8244 leader_election.cc:302] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 129b5aaf37054fc69f7936cf5b70f386, bd0f29970cc94da4b18a134fb024f348; no voters:
I20260806 01:00:42.970423 8882 raft_consensus.cc:2781] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:42.970645 8882 raft_consensus.cc:686] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 1 LEADER]: Becoming Leader. State: Replica: 129b5aaf37054fc69f7936cf5b70f386, State: Running, Role: LEADER
I20260806 01:00:42.970880 8882 consensus_queue.cc:237] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:42.973516 8104 catalog_manager.cc:5495] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 reported cstate change: term changed from 0 to 1, leader changed from <none> to 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129). New cstate: current_term: 1 leader_uuid: "129b5aaf37054fc69f7936cf5b70f386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } health_report { overall_health: UNKNOWN } } }
I20260806 01:00:43.008632 9022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:43.002194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38884 (local address 127.5.108.129:42091)
0806 01:00:43.003130 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:43.003134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:43.003180 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:00:43.003249 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:43.003253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:43.003312 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:43.003472 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:43.003949 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:43.005071 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:43.006551 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:43.007104 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:43.007219 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:43.007919 (+ 700us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:43.008041 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:43.008113 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:43.008244 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 01:00:43.008302 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":712,"threads_started":1}
I20260806 01:00:43.018137 8777 raft_consensus.cc:1261] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Refusing update from remote peer 129b5aaf37054fc69f7936cf5b70f386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:43.018332 8572 raft_consensus.cc:1261] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Refusing update from remote peer 129b5aaf37054fc69f7936cf5b70f386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:43.018782 8965 consensus_queue.cc:1035] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:43.019146 8985 consensus_queue.cc:1035] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:43.067989 8104 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:43.068171 8104 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:43.070901 8104 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac: Sending DeleteTablet for 3 replicas of tablet 078e106eeb9642eb9c5f5dcd6fb279fb
I20260806 01:00:43.071609 8332 tablet_service.cc:1514] Processing DeleteTablet for tablet 078e106eeb9642eb9c5f5dcd6fb279fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:43 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:40480
I20260806 01:00:43.071776 8746 tablet_service.cc:1514] Processing DeleteTablet for tablet 078e106eeb9642eb9c5f5dcd6fb279fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:43 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:37672
I20260806 01:00:43.071801 8547 tablet_service.cc:1514] Processing DeleteTablet for tablet 078e106eeb9642eb9c5f5dcd6fb279fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:43 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:60372
I20260806 01:00:43.072077 8995 tablet_replica.cc:330] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: stopping tablet replica
I20260806 01:00:43.072080 8992 tablet_replica.cc:330] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: stopping tablet replica
I20260806 01:00:43.072288 8995 raft_consensus.cc:2229] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:43.072284 8992 raft_consensus.cc:2229] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:00:43.072463 8995 raft_consensus.cc:2258] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:43.072587 8992 raft_consensus.cc:2258] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:43.072868 8993 tablet_replica.cc:330] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: stopping tablet replica
I20260806 01:00:43.073086 8993 raft_consensus.cc:2229] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:43.073266 8993 raft_consensus.cc:2258] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:43.073410 8995 ts_tablet_manager.cc:1905] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:43.074355 8993 ts_tablet_manager.cc:1905] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:43.079965 8995 ts_tablet_manager.cc:1918] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:00:43.080173 8992 ts_tablet_manager.cc:1905] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:43.080173 8995 log.cc:1196] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/078e106eeb9642eb9c5f5dcd6fb279fb
I20260806 01:00:43.080652 8995 ts_tablet_manager.cc:1939] T 078e106eeb9642eb9c5f5dcd6fb279fb P 5ed2c8fcac404bf3a78b789a2b277978: Deleting consensus metadata
I20260806 01:00:43.081744 8087 catalog_manager.cc:4841] TS 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841): tablet 078e106eeb9642eb9c5f5dcd6fb279fb (table test-table [id=6c97883cba1f499fb62717442f17dcee]) successfully deleted
I20260806 01:00:43.084987 8993 ts_tablet_manager.cc:1918] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:00:43.085131 8993 log.cc:1196] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/078e106eeb9642eb9c5f5dcd6fb279fb
I20260806 01:00:43.085521 8993 ts_tablet_manager.cc:1939] T 078e106eeb9642eb9c5f5dcd6fb279fb P bd0f29970cc94da4b18a134fb024f348: Deleting consensus metadata
I20260806 01:00:43.086594 8088 catalog_manager.cc:4841] TS bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231): tablet 078e106eeb9642eb9c5f5dcd6fb279fb (table test-table [id=6c97883cba1f499fb62717442f17dcee]) successfully deleted
I20260806 01:00:43.087206 8992 ts_tablet_manager.cc:1918] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:00:43.087411 8992 log.cc:1196] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/078e106eeb9642eb9c5f5dcd6fb279fb
I20260806 01:00:43.087795 8992 ts_tablet_manager.cc:1939] T 078e106eeb9642eb9c5f5dcd6fb279fb P 129b5aaf37054fc69f7936cf5b70f386: Deleting consensus metadata
I20260806 01:00:43.087693 8104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:00:43.088276 8104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:00:43.088644 8085 catalog_manager.cc:4841] TS 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129:42091): tablet 078e106eeb9642eb9c5f5dcd6fb279fb (table test-table [id=6c97883cba1f499fb62717442f17dcee]) successfully deleted
I20260806 01:00:43.094528 8332 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc2dc6c667640198264eebb1d1cded3 (DEFAULT_TABLE table=test-table [id=a3f16b1079e84e00b4851c559fbf139c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:43.094942 8746 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc2dc6c667640198264eebb1d1cded3 (DEFAULT_TABLE table=test-table [id=a3f16b1079e84e00b4851c559fbf139c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:43.095011 8332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bc2dc6c667640198264eebb1d1cded3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:43.094941 8547 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc2dc6c667640198264eebb1d1cded3 (DEFAULT_TABLE table=test-table [id=a3f16b1079e84e00b4851c559fbf139c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:00:43.095288 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bc2dc6c667640198264eebb1d1cded3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:43.095321 8547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bc2dc6c667640198264eebb1d1cded3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:43.099884 9009 tablet_bootstrap.cc:492] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: Bootstrap starting.
I20260806 01:00:43.101277 9009 tablet_bootstrap.cc:654] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:43.101655 9008 tablet_bootstrap.cc:492] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: Bootstrap starting.
I20260806 01:00:43.102046 9014 tablet_bootstrap.cc:492] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: Bootstrap starting.
I20260806 01:00:43.103315 9014 tablet_bootstrap.cc:654] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:43.104527 9008 tablet_bootstrap.cc:654] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:43.106000 9014 tablet_bootstrap.cc:492] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: No bootstrap required, opened a new log
I20260806 01:00:43.106182 9014 ts_tablet_manager.cc:1397] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260806 01:00:43.106436 9009 tablet_bootstrap.cc:492] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: No bootstrap required, opened a new log
I20260806 01:00:43.106621 9009 ts_tablet_manager.cc:1397] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 01:00:43.106958 9008 tablet_bootstrap.cc:492] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: No bootstrap required, opened a new log
I20260806 01:00:43.106895 9014 raft_consensus.cc:348] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:43.107117 9008 ts_tablet_manager.cc:1397] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 01:00:43.107132 9014 raft_consensus.cc:374] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:43.107220 9014 raft_consensus.cc:729] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0f29970cc94da4b18a134fb024f348, State: Initialized, Role: FOLLOWER
I20260806 01:00:43.107414 9009 raft_consensus.cc:348] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:43.107491 9014 consensus_queue.cc:260] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:43.107671 9009 raft_consensus.cc:374] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:43.107757 9009 raft_consensus.cc:729] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed2c8fcac404bf3a78b789a2b277978, State: Initialized, Role: FOLLOWER
I20260806 01:00:43.108009 9014 ts_tablet_manager.cc:1428] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 01:00:43.107898 9008 raft_consensus.cc:348] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:43.107990 9009 consensus_queue.cc:260] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:43.108121 9008 raft_consensus.cc:374] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:43.108191 9008 raft_consensus.cc:729] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 129b5aaf37054fc69f7936cf5b70f386, State: Initialized, Role: FOLLOWER
I20260806 01:00:43.108527 9009 ts_tablet_manager.cc:1428] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 01:00:43.108425 9008 consensus_queue.cc:260] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:43.109091 9008 ts_tablet_manager.cc:1428] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 01:00:43.200888 9018 raft_consensus.cc:482] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:00:43.201171 9018 raft_consensus.cc:504] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:43.202188 9018 leader_election.cc:288] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129:42091), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:43.202672 8368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc2dc6c667640198264eebb1d1cded3" candidate_uuid: "bd0f29970cc94da4b18a134fb024f348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "129b5aaf37054fc69f7936cf5b70f386" is_pre_election: true
I20260806 01:00:43.202710 8776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc2dc6c667640198264eebb1d1cded3" candidate_uuid: "bd0f29970cc94da4b18a134fb024f348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978" is_pre_election: true
I20260806 01:00:43.202968 8776 raft_consensus.cc:2454] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd0f29970cc94da4b18a134fb024f348 in term 0.
I20260806 01:00:43.202968 8368 raft_consensus.cc:2454] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd0f29970cc94da4b18a134fb024f348 in term 0.
I20260806 01:00:43.203474 8475 leader_election.cc:302] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 129b5aaf37054fc69f7936cf5b70f386, 5ed2c8fcac404bf3a78b789a2b277978, bd0f29970cc94da4b18a134fb024f348; no voters:
I20260806 01:00:43.203785 9018 raft_consensus.cc:2781] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:00:43.203914 9018 raft_consensus.cc:482] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:00:43.204010 9018 raft_consensus.cc:3037] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:43.206513 9018 raft_consensus.cc:504] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:43.207006 9018 leader_election.cc:288] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [CANDIDATE]: Term 1 election: Requested vote from peers 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129:42091), 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841)
I20260806 01:00:43.207578 8776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc2dc6c667640198264eebb1d1cded3" candidate_uuid: "bd0f29970cc94da4b18a134fb024f348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2c8fcac404bf3a78b789a2b277978"
I20260806 01:00:43.207579 8368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc2dc6c667640198264eebb1d1cded3" candidate_uuid: "bd0f29970cc94da4b18a134fb024f348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "129b5aaf37054fc69f7936cf5b70f386"
I20260806 01:00:43.207777 8776 raft_consensus.cc:3037] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:43.207830 8368 raft_consensus.cc:3037] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:43.211158 8776 raft_consensus.cc:2454] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd0f29970cc94da4b18a134fb024f348 in term 1.
I20260806 01:00:43.211464 8368 raft_consensus.cc:2454] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd0f29970cc94da4b18a134fb024f348 in term 1.
I20260806 01:00:43.211611 8475 leader_election.cc:302] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ed2c8fcac404bf3a78b789a2b277978, bd0f29970cc94da4b18a134fb024f348; no voters:
I20260806 01:00:43.211938 9018 raft_consensus.cc:2781] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:43.212546 9018 raft_consensus.cc:686] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 1 LEADER]: Becoming Leader. State: Replica: bd0f29970cc94da4b18a134fb024f348, State: Running, Role: LEADER
I20260806 01:00:43.213022 9018 consensus_queue.cc:237] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } }
I20260806 01:00:43.215763 8100 catalog_manager.cc:5495] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd0f29970cc94da4b18a134fb024f348 (127.5.108.130). New cstate: current_term: 1 leader_uuid: "bd0f29970cc94da4b18a134fb024f348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd0f29970cc94da4b18a134fb024f348" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 } health_report { overall_health: UNKNOWN } } }
I20260806 01:00:43.245158 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:43.237890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.5.108.130:39231)
0806 01:00:43.239084 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:43.239091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:43.239128 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:00:43.239180 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:43.239186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:43.239244 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:43.239414 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:00:43.239927 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:43.241091 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:43.242773 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:43.243528 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:43.243670 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:43.244234 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:00:43.244359 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:00:43.244486 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:43.244659 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 01:00:43.244735 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":996,"thread_start_us":836,"threads_started":1}
I20260806 01:00:43.254041 8776 raft_consensus.cc:1261] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Refusing update from remote peer bd0f29970cc94da4b18a134fb024f348: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:43.254098 8368 raft_consensus.cc:1261] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Refusing update from remote peer bd0f29970cc94da4b18a134fb024f348: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:00:43.254643 9038 consensus_queue.cc:1035] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ed2c8fcac404bf3a78b789a2b277978" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 46841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:00:43.255180 9018 consensus_queue.cc:1035] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "129b5aaf37054fc69f7936cf5b70f386" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 42091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:00:43.275082 8967 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 01:00:44.853641 8101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:44.853977 8101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:00:44.857970 8101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c846a7944ebe4a8dad7a78e6b8a981ac: Sending DeleteTablet for 3 replicas of tablet 3bc2dc6c667640198264eebb1d1cded3
I20260806 01:00:44.858642 8332 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bc2dc6c667640198264eebb1d1cded3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:44 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:40480
I20260806 01:00:44.858759 8547 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bc2dc6c667640198264eebb1d1cded3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:44 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:60372
I20260806 01:00:44.858958 8746 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bc2dc6c667640198264eebb1d1cded3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:00:44 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:37672
I20260806 01:00:44.859503 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 8219
I20260806 01:00:44.870275 9084 tablet_replica.cc:330] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: stopping tablet replica
I20260806 01:00:44.870478 9080 tablet_replica.cc:330] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: stopping tablet replica
I20260806 01:00:44.870508 9084 raft_consensus.cc:2229] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:44.870695 9080 raft_consensus.cc:2229] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:00:44.870771 9084 raft_consensus.cc:2258] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:44.871011 9080 raft_consensus.cc:2258] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:44.871332 9082 tablet_replica.cc:330] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: stopping tablet replica
I20260806 01:00:44.871655 9082 raft_consensus.cc:2229] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:00:44.872041 9082 raft_consensus.cc:2258] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:00:44.872539 9084 ts_tablet_manager.cc:1905] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:44.872563 9080 ts_tablet_manager.cc:1905] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:44.872928 9082 ts_tablet_manager.cc:1905] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:00:44.879614 9084 ts_tablet_manager.cc:1918] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:00:44.879769 9084 log.cc:1196] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/3bc2dc6c667640198264eebb1d1cded3
I20260806 01:00:44.880452 9084 ts_tablet_manager.cc:1939] T 3bc2dc6c667640198264eebb1d1cded3 P 5ed2c8fcac404bf3a78b789a2b277978: Deleting consensus metadata
I20260806 01:00:44.880520 9082 ts_tablet_manager.cc:1918] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:00:44.880702 9082 log.cc:1196] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/3bc2dc6c667640198264eebb1d1cded3
I20260806 01:00:44.880821 9080 ts_tablet_manager.cc:1918] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:00:44.881058 9080 log.cc:1196] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/3bc2dc6c667640198264eebb1d1cded3
I20260806 01:00:44.881349 9082 ts_tablet_manager.cc:1939] T 3bc2dc6c667640198264eebb1d1cded3 P bd0f29970cc94da4b18a134fb024f348: Deleting consensus metadata
I20260806 01:00:44.881795 9080 ts_tablet_manager.cc:1939] T 3bc2dc6c667640198264eebb1d1cded3 P 129b5aaf37054fc69f7936cf5b70f386: Deleting consensus metadata
I20260806 01:00:44.881881 8087 catalog_manager.cc:4841] TS 5ed2c8fcac404bf3a78b789a2b277978 (127.5.108.131:46841): tablet 3bc2dc6c667640198264eebb1d1cded3 (table test-table [id=a3f16b1079e84e00b4851c559fbf139c]) successfully deleted
I20260806 01:00:44.882510 8088 catalog_manager.cc:4841] TS bd0f29970cc94da4b18a134fb024f348 (127.5.108.130:39231): tablet 3bc2dc6c667640198264eebb1d1cded3 (table test-table [id=a3f16b1079e84e00b4851c559fbf139c]) successfully deleted
I20260806 01:00:44.883353 8085 catalog_manager.cc:4841] TS 129b5aaf37054fc69f7936cf5b70f386 (127.5.108.129:42091): tablet 3bc2dc6c667640198264eebb1d1cded3 (table test-table [id=a3f16b1079e84e00b4851c559fbf139c]) successfully deleted
Aug 06 01:00:44 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:00:44.894182 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:44.861283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38892 (local address 127.5.108.129:42091)
0806 01:00:44.871489 (+ 10206us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:44.871496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:44.871543 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:00:44.871620 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:44.871625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:44.871700 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:44.871816 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:44.873748 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:44.874987 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:44.876890 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:44.877582 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:44.877713 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:44.887661 (+ 9948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:44.887689 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:44.887706 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:44.887776 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:44.891077 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:44.891744 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:44.891751 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:44.891753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:44.892045 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:44.892479 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:44.892482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:44.892484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:44.892912 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:44.893184 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:44.893502 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 01:00:44.893743 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10035,"thread_start_us":9434,"threads_started":1}
W20260806 01:00:44.894981 8431 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38892
W20260806 01:00:44.895706 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 8219: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:00:44.896034 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 8219
I20260806 01:00:44.932063 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 8447
Aug 06 01:00:44 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:00:44.947758 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:44.932873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38028 (local address 127.5.108.130:39231)
0806 01:00:44.933766 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:44.933770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:44.933787 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:00:44.933843 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:44.933846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:44.933909 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:44.934001 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:44.935210 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:44.936190 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:44.937333 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:44.937733 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:44.937827 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:44.942294 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:44.942307 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:44.942323 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:44.942354 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:44.945299 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:44.945980 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:44.945986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:44.946006 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:44.946143 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:44.946481 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:44.946484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:44.946485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:44.946818 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:44.947027 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:44.947256 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 01:00:44.947473 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":601,"threads_started":1}
W20260806 01:00:44.948436 8647 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38028
W20260806 01:00:44.949077 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 8447: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:00:44.949249 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 8447
I20260806 01:00:44.980098 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 8658
Aug 06 01:00:44 dist-test-slave-x4qh krb5kdc[8040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:00:44.994912 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:44.980818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57072 (local address 127.5.108.131:46841)
0806 01:00:44.981368 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:44.981372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:44.981388 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:00:44.981443 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:44.981446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:44.981503 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:44.981593 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:44.982828 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:44.983656 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:44.984855 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:44.985298 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:44.985387 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:44.989575 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:44.989587 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:44.989602 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:44.989632 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:44.992646 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:44.993243 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:44.993250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:44.993252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:44.993379 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:44.993691 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:44.993694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:44.993696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:44.994026 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:44.994257 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:44.994469 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 01:00:44.994616 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":290,"threads_started":1}
W20260806 01:00:44.995648 8838 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57072
W20260806 01:00:44.996268 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 8658: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:00:44.996434 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 8658
I20260806 01:00:45.024171 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 8061
I20260806 01:00:45.025345 8174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:45.139420 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 8061
2026-08-06T01:00:45Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8906 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:00:45 dist-test-slave-x4qh krb5kdc[9093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:00:45 dist-test-slave-x4qh krb5kdc[9093](info): set up 2 sockets
Aug 06 01:00:45 dist-test-slave-x4qh krb5kdc[9093](info): commencing operation
krb5kdc: starting...
W20260806 01:00:46.396068 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:00:46 dist-test-slave-x4qh krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:00:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:00:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:00:46.586781 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34429
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:34429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:46.910262 9117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:46.910600 9117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:46.910786 9117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:46.917811 9117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:00:46.917948 9117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:46.918071 9117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:46.918175 9117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:00:46.918272 9117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:00:46.927837 9117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:34429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:46.929299 9117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:46.930662 9117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:46.942415 9124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:46.942447 9125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:46.942397 9128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:46.943595 9117 server_base.cc:1034] running on GCE node
I20260806 01:00:46.944387 9117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:46.945706 9117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:46.946938 9117 hybrid_clock.cc:648] HybridClock initialized: now 1785978046946892 us; error 53 us; skew 500 ppm
I20260806 01:00:46.947283 9117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:46 dist-test-slave-x4qh krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978046, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:46.950281 9117 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:00:46.951810 9117 webserver.cc:460] Webserver started at http://127.5.108.190:46187/ using document root <none> and password file <none>
I20260806 01:00:46.952492 9117 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:46.952641 9117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:46.952960 9117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:46.955977 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "8e9b646f71144e9a95f4b1dbfd85db2f"
format_stamp: "Formatted at 2026-08-06 01:00:46 on dist-test-slave-x4qh"
server_key: "7d58023a1262ae1d13a19faaaad38704"
server_key_iv: "522eaca598c19e3830d3127951ddd124"
server_key_version: "encryptionkey@0"
I20260806 01:00:46.956754 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "8e9b646f71144e9a95f4b1dbfd85db2f"
format_stamp: "Formatted at 2026-08-06 01:00:46 on dist-test-slave-x4qh"
server_key: "7d58023a1262ae1d13a19faaaad38704"
server_key_iv: "522eaca598c19e3830d3127951ddd124"
server_key_version: "encryptionkey@0"
I20260806 01:00:46.961619 9117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:00:46.964931 9137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:46.966161 9117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:00:46.966351 9117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "8e9b646f71144e9a95f4b1dbfd85db2f"
format_stamp: "Formatted at 2026-08-06 01:00:46 on dist-test-slave-x4qh"
server_key: "7d58023a1262ae1d13a19faaaad38704"
server_key_iv: "522eaca598c19e3830d3127951ddd124"
server_key_version: "encryptionkey@0"
I20260806 01:00:46.966552 9117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:46.977298 9117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:46.980986 9117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:46.981292 9117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:46.999682 9117 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:34429
I20260806 01:00:46.999713 9206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:34429 every 8 connection(s)
I20260806 01:00:47.000978 9117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:00:47.004446 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 9117
I20260806 01:00:47.004499 9207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:47.004714 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:00:47.005345 5554 external_mini_cluster.cc:1428] Setting key 5772281038488437398bb58080f9ad2e
I20260806 01:00:47.018328 9207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f: Bootstrap starting.
Aug 06 01:00:47 dist-test-slave-x4qh krb5kdc[9093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:47.021490 9207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:47.022854 9207 log.cc:824] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:47.026163 9207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f: No bootstrap required, opened a new log
I20260806 01:00:47.029791 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:47.007216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45770 (local address 127.5.108.190:34429)
0806 01:00:47.008091 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:47.008106 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:47.008157 (+ 51us) server_negotiation.cc:407] Connection header received
0806 01:00:47.010949 (+ 2792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:47.010976 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:47.011482 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:47.011893 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:47.013231 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:47.015128 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:47.016429 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:47.017418 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:47.017589 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:47.021504 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:47.021518 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:47.021536 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:47.021580 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:47.025629 (+ 4049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:47.026223 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:47.026230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:47.026245 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:47.026453 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:47.026803 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:47.026808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:47.026811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:47.027501 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:47.027781 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:47.028120 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 01:00:47.028537 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":347,"threads_started":1}
I20260806 01:00:47.035169 9207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9b646f71144e9a95f4b1dbfd85db2f" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34429 } }
I20260806 01:00:47.035542 9207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:47.035624 9207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9b646f71144e9a95f4b1dbfd85db2f, State: Initialized, Role: FOLLOWER
I20260806 01:00:47.036165 9207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9b646f71144e9a95f4b1dbfd85db2f" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34429 } }
I20260806 01:00:47.036374 9207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:00:47.036470 9207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:00:47.036592 9207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:47.038810 9207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9b646f71144e9a95f4b1dbfd85db2f" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34429 } }
I20260806 01:00:47.039217 9207 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e9b646f71144e9a95f4b1dbfd85db2f; no voters:
I20260806 01:00:47.039770 9207 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:00:47.039934 9214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:47.040619 9214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [term 1 LEADER]: Becoming Leader. State: Replica: 8e9b646f71144e9a95f4b1dbfd85db2f, State: Running, Role: LEADER
I20260806 01:00:47.041173 9214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9b646f71144e9a95f4b1dbfd85db2f" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34429 } }
I20260806 01:00:47.041839 9207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:00:47.044090 9215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e9b646f71144e9a95f4b1dbfd85db2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9b646f71144e9a95f4b1dbfd85db2f" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34429 } } }
I20260806 01:00:47.044503 9215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:47.044495 9216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9b646f71144e9a95f4b1dbfd85db2f. Latest consensus state: current_term: 1 leader_uuid: "8e9b646f71144e9a95f4b1dbfd85db2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9b646f71144e9a95f4b1dbfd85db2f" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34429 } } }
I20260806 01:00:47.044922 9216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:47.046471 9230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:00:47.051206 9230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:00:47.058836 9230 catalog_manager.cc:1347] Generated new cluster ID: 62f7bea967664bcbb3621e7861286772
I20260806 01:00:47.058943 9230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:00:47.068748 9230 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:00:47.070533 9230 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:00:47.078801 9230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e9b646f71144e9a95f4b1dbfd85db2f: Generated new TSK 0
I20260806 01:00:47.079785 9230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:00:47.136543 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34429
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:47.476914 9245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:47.477172 9245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:47.477257 9245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:47.484313 9245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:47.484427 9245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:47.484536 9245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:00:47.493820 9245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:47.494858 9245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:47.496114 9245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:47.508780 9253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:47.509011 9255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:47.509747 9257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:47.510623 9245 server_base.cc:1034] running on GCE node
I20260806 01:00:47.511358 9245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:47.512564 9245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:47.513808 9245 hybrid_clock.cc:648] HybridClock initialized: now 1785978047513754 us; error 38 us; skew 500 ppm
I20260806 01:00:47.514163 9245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:47 dist-test-slave-x4qh krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978047, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:47.517985 9245 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:00:47.519315 9245 webserver.cc:460] Webserver started at http://127.5.108.129:44923/ using document root <none> and password file <none>
I20260806 01:00:47.519984 9245 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:47.520121 9245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:47.520457 9245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:47.523229 9245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "c6d8812d93da4a0c9b8dfffcee88cc8d"
format_stamp: "Formatted at 2026-08-06 01:00:47 on dist-test-slave-x4qh"
server_key: "dde258d16bb3d154a59d0f7ac5ea5aec"
server_key_iv: "0f3e90b5dce83d1febb9fe1b4efc6ed5"
server_key_version: "encryptionkey@0"
I20260806 01:00:47.524075 9245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "c6d8812d93da4a0c9b8dfffcee88cc8d"
format_stamp: "Formatted at 2026-08-06 01:00:47 on dist-test-slave-x4qh"
server_key: "dde258d16bb3d154a59d0f7ac5ea5aec"
server_key_iv: "0f3e90b5dce83d1febb9fe1b4efc6ed5"
server_key_version: "encryptionkey@0"
I20260806 01:00:47.528963 9245 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:00:47.532218 9265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:47.533468 9245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 01:00:47.533648 9245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "c6d8812d93da4a0c9b8dfffcee88cc8d"
format_stamp: "Formatted at 2026-08-06 01:00:47 on dist-test-slave-x4qh"
server_key: "dde258d16bb3d154a59d0f7ac5ea5aec"
server_key_iv: "0f3e90b5dce83d1febb9fe1b4efc6ed5"
server_key_version: "encryptionkey@0"
I20260806 01:00:47.533833 9245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:47.562655 9245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:47.566620 9245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:47.566913 9245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:47.567971 9245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:47.569284 9245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:47.569356 9245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:47.569519 9245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:47.569585 9245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:47.602972 9245 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:46045
I20260806 01:00:47.603020 9439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:46045 every 8 connection(s)
I20260806 01:00:47.604286 9245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:00:47.605626 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 9245
I20260806 01:00:47.605873 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:00:47.606475 5554 external_mini_cluster.cc:1428] Setting key f7c872fb4199fb7e8fb72550efc070c6
Aug 06 01:00:47 dist-test-slave-x4qh krb5kdc[9093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978047, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:47.627997 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:47.606976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:52445 (local address 127.5.108.190:34429)
0806 01:00:47.607470 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:47.607475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:47.608450 (+ 975us) server_negotiation.cc:407] Connection header received
0806 01:00:47.611306 (+ 2856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:47.611310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:47.611431 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:47.611532 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:47.613618 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:47.614258 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:47.616609 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:47.617137 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:47.617291 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:47.620788 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:47.620797 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:47.620804 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:47.620854 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:47.624718 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:47.625399 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:47.625408 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:47.625414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:47.625569 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:47.625991 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:47.625998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:47.626004 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:47.626412 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:47.626573 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:47.627235 (+ 662us) server_negotiation.cc:299] Negotiation successful
0806 01:00:47.627523 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":249,"threads_started":1}
I20260806 01:00:47.628533 9445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:47.607528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34429 (local address 127.5.108.129:52445)
0806 01:00:47.608276 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:47.608357 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:47.611028 (+ 2671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:47.611805 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:47.611818 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:47.612607 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:47.613420 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:47.613448 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:47.614447 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:47.614454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:47.616478 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:47.616494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:47.617380 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:47.617384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:47.617568 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:47.618241 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:47.618263 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:47.620560 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:47.624863 (+ 4303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:47.624869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:47.624880 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:47.625265 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:47.625677 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:47.625680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:47.625682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:47.625860 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:47.626554 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:47.626559 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:47.626849 (+ 290us) client_negotiation.cc:769] Sending connection context
0806 01:00:47.627050 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 01:00:47.627475 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":330,"threads_started":1}
I20260806 01:00:47.630051 9440 heartbeater.cc:347] Connected to a master server at 127.5.108.190:34429
I20260806 01:00:47.630358 9440 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
I20260806 01:00:47.631120 9440 heartbeater.cc:511] Master 127.5.108.190:34429 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
I20260806 01:00:47.633404 9158 ts_manager.cc:194] Registered new tserver with Master: c6d8812d93da4a0c9b8dfffcee88cc8d (127.5.108.129:46045)
I20260806 01:00:47.635692 9158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:52445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:00:47.658429 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34429
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:47.987149 9450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:47.987430 9450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:47.987525 9450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:47.994620 9450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:47.994727 9450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:47.994836 9450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:00:48.004555 9450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:48.005774 9450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:48.007088 9450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:48.020390 9460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:48.020444 9456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:48.020846 9464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:48.021414 9450 server_base.cc:1034] running on GCE node
I20260806 01:00:48.022001 9450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:48.022897 9450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:48.024103 9450 hybrid_clock.cc:648] HybridClock initialized: now 1785978048024072 us; error 32 us; skew 500 ppm
I20260806 01:00:48.024389 9450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:48 dist-test-slave-x4qh krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978048, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:48.027539 9450 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:00:48.029387 9450 webserver.cc:460] Webserver started at http://127.5.108.130:33869/ using document root <none> and password file <none>
I20260806 01:00:48.030071 9450 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:48.030215 9450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:48.030529 9450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:48.033355 9450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "419f8e06b6ce440da5cb7b628fa718be"
format_stamp: "Formatted at 2026-08-06 01:00:48 on dist-test-slave-x4qh"
server_key: "bee3a96347e57e03782e4768bb6f260f"
server_key_iv: "cca8cd3d42d04b25a222238772492673"
server_key_version: "encryptionkey@0"
I20260806 01:00:48.034157 9450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "419f8e06b6ce440da5cb7b628fa718be"
format_stamp: "Formatted at 2026-08-06 01:00:48 on dist-test-slave-x4qh"
server_key: "bee3a96347e57e03782e4768bb6f260f"
server_key_iv: "cca8cd3d42d04b25a222238772492673"
server_key_version: "encryptionkey@0"
I20260806 01:00:48.039425 9450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:00:48.042910 9474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:48.044210 9450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:00:48.044432 9450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "419f8e06b6ce440da5cb7b628fa718be"
format_stamp: "Formatted at 2026-08-06 01:00:48 on dist-test-slave-x4qh"
server_key: "bee3a96347e57e03782e4768bb6f260f"
server_key_iv: "cca8cd3d42d04b25a222238772492673"
server_key_version: "encryptionkey@0"
I20260806 01:00:48.044629 9450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:48.065714 9450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:48.069854 9450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:48.070134 9450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:48.071159 9450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:48.072458 9450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:48.072566 9450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:48.072719 9450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:48.072803 9450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:48.108788 9450 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:35633
I20260806 01:00:48.108841 9645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:35633 every 8 connection(s)
I20260806 01:00:48.110138 9450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:00:48.117738 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 9450
I20260806 01:00:48.117972 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:00:48.118444 5554 external_mini_cluster.cc:1428] Setting key 94c983496dcf542952046d4291450c25
Aug 06 01:00:48 dist-test-slave-x4qh krb5kdc[9093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978048, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:48.129760 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:48.112788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:43995 (local address 127.5.108.190:34429)
0806 01:00:48.112983 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:48.112987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:48.114060 (+ 1073us) server_negotiation.cc:407] Connection header received
0806 01:00:48.115872 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:48.115878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:48.115972 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:48.116087 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:48.117534 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:48.118359 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:48.120168 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:48.120707 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:48.120836 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:48.124465 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:48.124470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:48.124473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:48.124502 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:48.126918 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:48.127636 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:48.127643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:48.127650 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:48.127766 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:48.128171 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:48.128181 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:48.128183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:48.128444 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:48.128599 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:48.129237 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 01:00:48.129442 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:00:48.130682 9649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:48.113201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34429 (local address 127.5.108.130:43995)
0806 01:00:48.113909 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:48.113953 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:48.115584 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:48.116305 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:48.116313 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:48.116823 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:48.117346 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:48.117384 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:48.118552 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:48.118559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:48.120034 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:48.120046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:48.120945 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:48.120952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:48.121223 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:48.121933 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:48.121954 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:48.124245 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:48.127052 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:48.127060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:48.127073 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:48.127522 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:48.127876 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:48.127881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:48.127884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:48.128052 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:48.128573 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:48.128581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:48.128896 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 01:00:48.129141 (+ 245us) client_negotiation.cc:240] Negotiation successful
0806 01:00:48.129599 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":299,"threads_started":1}
I20260806 01:00:48.132189 9647 heartbeater.cc:347] Connected to a master server at 127.5.108.190:34429
I20260806 01:00:48.132555 9647 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:48.133456 9647 heartbeater.cc:511] Master 127.5.108.190:34429 requested a full tablet report, sending...
I20260806 01:00:48.135250 9158 ts_manager.cc:194] Registered new tserver with Master: 419f8e06b6ce440da5cb7b628fa718be (127.5.108.130:35633)
I20260806 01:00:48.136634 9158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:43995
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:00:48.167980 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34429
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:48.487020 9653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:48.487293 9653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:48.487435 9653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:48.494144 9653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:48.494230 9653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:48.494342 9653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:00:48.503139 9653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:48.504297 9653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:48.505649 9653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:48.518123 9663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:48.518239 9664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:48.518276 9666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:48.519093 9653 server_base.cc:1034] running on GCE node
I20260806 01:00:48.519706 9653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:48.520547 9653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:48.521744 9653 hybrid_clock.cc:648] HybridClock initialized: now 1785978048521690 us; error 51 us; skew 500 ppm
I20260806 01:00:48.522042 9653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:48 dist-test-slave-x4qh krb5kdc[9093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978048, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:48.525257 9653 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:00:48.526584 9653 webserver.cc:460] Webserver started at http://127.5.108.131:41151/ using document root <none> and password file <none>
I20260806 01:00:48.527231 9653 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:48.527410 9653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:48.527774 9653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:48.530244 9653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "9453d9a505a341a9b5a12f9d2d350a00"
format_stamp: "Formatted at 2026-08-06 01:00:48 on dist-test-slave-x4qh"
server_key: "43f643a03729f537bd137605885a21ee"
server_key_iv: "37813aa416456343d04af8c8a70086be"
server_key_version: "encryptionkey@0"
I20260806 01:00:48.530920 9653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "9453d9a505a341a9b5a12f9d2d350a00"
format_stamp: "Formatted at 2026-08-06 01:00:48 on dist-test-slave-x4qh"
server_key: "43f643a03729f537bd137605885a21ee"
server_key_iv: "37813aa416456343d04af8c8a70086be"
server_key_version: "encryptionkey@0"
I20260806 01:00:48.535557 9653 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.002s
I20260806 01:00:48.538800 9677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:48.540084 9653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:00:48.540225 9653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "9453d9a505a341a9b5a12f9d2d350a00"
format_stamp: "Formatted at 2026-08-06 01:00:48 on dist-test-slave-x4qh"
server_key: "43f643a03729f537bd137605885a21ee"
server_key_iv: "37813aa416456343d04af8c8a70086be"
server_key_version: "encryptionkey@0"
I20260806 01:00:48.540372 9653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:48.573807 9653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:48.577823 9653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:48.578058 9653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:48.578927 9653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:48.580274 9653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:48.580348 9653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:48.580436 9653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:48.580479 9653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:48.613911 9653 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:44267
I20260806 01:00:48.613992 9833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:44267 every 8 connection(s)
I20260806 01:00:48.615046 9653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:00:48.615630 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 9653
I20260806 01:00:48.615856 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:00:48.616365 5554 external_mini_cluster.cc:1428] Setting key 69dc698a1d03df1d97395c2fa2700bc4
Aug 06 01:00:48 dist-test-slave-x4qh krb5kdc[9093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978048, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:48.635205 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:48.617711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:47303 (local address 127.5.108.190:34429)
0806 01:00:48.617924 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:48.617929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:48.619129 (+ 1200us) server_negotiation.cc:407] Connection header received
0806 01:00:48.620965 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:48.620970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:48.621084 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:48.621212 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:48.622632 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:48.623495 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:48.625114 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:48.625514 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:48.625647 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:48.628956 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:48.628962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:48.628967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:48.629007 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:48.632209 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:48.632927 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:48.632934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:48.632941 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:48.633078 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:48.633508 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:48.633514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:48.633517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:48.633805 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:48.633959 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:48.634602 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 01:00:48.634815 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:00:48.636143 9839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:48.618221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34429 (local address 127.5.108.131:47303)
0806 01:00:48.618982 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:48.619021 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:48.620726 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:48.621449 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:48.621457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:48.621928 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:48.622442 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:48.622477 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:48.623666 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:48.623670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:48.624967 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:48.624980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:48.625749 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:48.625756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:48.625973 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:48.626632 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:48.626666 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:48.628768 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:48.632340 (+ 3572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:48.632348 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:48.632361 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:48.632797 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:48.633201 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:48.633207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:48.633211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:48.633389 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:48.633935 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:48.633942 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:48.634252 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 01:00:48.634517 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 01:00:48.635026 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":369,"threads_started":1}
I20260806 01:00:48.637531 9834 heartbeater.cc:347] Connected to a master server at 127.5.108.190:34429
I20260806 01:00:48.637842 9834 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:48.638540 9834 heartbeater.cc:511] Master 127.5.108.190:34429 requested a full tablet report, sending...
I20260806 01:00:48.638829 9440 heartbeater.cc:503] Master 127.5.108.190:34429 was elected leader, sending a full tablet report...
I20260806 01:00:48.640158 9157 ts_manager.cc:194] Registered new tserver with Master: 9453d9a505a341a9b5a12f9d2d350a00 (127.5.108.131:44267)
I20260806 01:00:48.641000 9157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:47303
I20260806 01:00:48.644186 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 01:00:48.681788 9444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:00:48.679337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.5.108.190:34429)
0806 01:00:48.679610 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:48.679619 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:48.680004 (+ 385us) server_negotiation.cc:407] Connection header received
0806 01:00:48.680153 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:48.680157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:48.680226 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:48.680318 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:48.681495 (+ 1177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45780: BlockingRecv error: recv got EOF from 127.0.0.1:45780 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 01:00:48.685055 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 9245
W20260806 01:00:48.687970 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 9245: Not authorized: Client connection negotiation failed: client connection to 127.5.108.129:46045: server requires authentication, but client does not have Kerberos credentials available
I20260806 01:00:48.688148 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 9245
I20260806 01:00:48.711337 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 9450
W20260806 01:00:48.713966 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 9450: Not authorized: Client connection negotiation failed: client connection to 127.5.108.130:35633: server requires authentication, but client does not have Kerberos credentials available
I20260806 01:00:48.714124 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 9450
I20260806 01:00:48.736975 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 9653
W20260806 01:00:48.739915 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 9653: Not authorized: Client connection negotiation failed: client connection to 127.5.108.131:44267: server requires authentication, but client does not have Kerberos credentials available
I20260806 01:00:48.740048 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 9653
I20260806 01:00:48.762140 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 9117
I20260806 01:00:48.763356 9202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:48.864223 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 9117
2026-08-06T01:00:48Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3726 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:00:48 dist-test-slave-x4qh krb5kdc[9859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:00:48 dist-test-slave-x4qh krb5kdc[9859](info): set up 2 sockets
Aug 06 01:00:48 dist-test-slave-x4qh krb5kdc[9859](info): commencing operation
krb5kdc: starting...
W20260806 01:00:50.127485 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:00:50 dist-test-slave-x4qh krb5kdc[9859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:00:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:00:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:00:50.335779 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:33607
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:33607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:50.659639 9880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:50.659989 9880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:50.660177 9880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:50.667132 9880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:00:50.667277 9880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:50.667416 9880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:50.667510 9880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:00:50.667595 9880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:00:50.677217 9880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:33607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:33607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:50.678499 9880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:50.679867 9880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:50.691640 9896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:50.691728 9888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:50.691758 9892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:50.692538 9880 server_base.cc:1034] running on GCE node
I20260806 01:00:50.693447 9880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:50.694780 9880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:50.695984 9880 hybrid_clock.cc:648] HybridClock initialized: now 1785978050695948 us; error 33 us; skew 500 ppm
I20260806 01:00:50.696321 9880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:50 dist-test-slave-x4qh krb5kdc[9859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978050, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:50.702219 9880 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:00:50.703796 9880 webserver.cc:460] Webserver started at http://127.5.108.190:36545/ using document root <none> and password file <none>
I20260806 01:00:50.704474 9880 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:50.704639 9880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:50.704967 9880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:50.707944 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "91d4ffc00b6f4e04ad2572110f75457e"
format_stamp: "Formatted at 2026-08-06 01:00:50 on dist-test-slave-x4qh"
server_key: "5bba37a5c05736408500e5f6ca4e4817"
server_key_iv: "f7909c5dbbdc9a7ea0eb4fee2256228f"
server_key_version: "encryptionkey@0"
I20260806 01:00:50.708729 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "91d4ffc00b6f4e04ad2572110f75457e"
format_stamp: "Formatted at 2026-08-06 01:00:50 on dist-test-slave-x4qh"
server_key: "5bba37a5c05736408500e5f6ca4e4817"
server_key_iv: "f7909c5dbbdc9a7ea0eb4fee2256228f"
server_key_version: "encryptionkey@0"
I20260806 01:00:50.713647 9880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:00:50.716990 9907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:50.718288 9880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:00:50.718474 9880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "91d4ffc00b6f4e04ad2572110f75457e"
format_stamp: "Formatted at 2026-08-06 01:00:50 on dist-test-slave-x4qh"
server_key: "5bba37a5c05736408500e5f6ca4e4817"
server_key_iv: "f7909c5dbbdc9a7ea0eb4fee2256228f"
server_key_version: "encryptionkey@0"
I20260806 01:00:50.718663 9880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:50.738543 9880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:50.742620 9880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:50.742937 9880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:50.761396 9880 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:33607
I20260806 01:00:50.761431 9991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:33607 every 8 connection(s)
I20260806 01:00:50.762720 9880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:00:50.766185 9992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:50.767589 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 9880
I20260806 01:00:50.767812 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:00:50.768661 5554 external_mini_cluster.cc:1428] Setting key 71901d8fea7d1c6aaf2acfdce064623d
I20260806 01:00:50.778939 9992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e: Bootstrap starting.
Aug 06 01:00:50 dist-test-slave-x4qh krb5kdc[9859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:50.782186 9992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:50.783249 9992 log.cc:824] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:50.786017 9992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e: No bootstrap required, opened a new log
I20260806 01:00:50.790283 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:50.770579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.5.108.190:33607)
0806 01:00:50.771450 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:50.771462 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:50.771508 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:00:50.773077 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:50.773098 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:50.773498 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:50.773901 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:50.775449 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:50.776680 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:50.777909 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:50.778555 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:50.778756 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:50.783068 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:50.783075 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:50.783093 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:50.783129 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:50.786104 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:50.787012 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:50.787017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:50.787022 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:50.787173 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:50.787631 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:50.787635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:50.787637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:50.788167 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:50.788404 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:50.788855 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 01:00:50.789231 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":357,"threads_started":1}
I20260806 01:00:50.794030 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:00:50.794920 9992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d4ffc00b6f4e04ad2572110f75457e" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33607 } }
I20260806 01:00:50.795266 9992 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:50.795390 9992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91d4ffc00b6f4e04ad2572110f75457e, State: Initialized, Role: FOLLOWER
I20260806 01:00:50.795938 9992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d4ffc00b6f4e04ad2572110f75457e" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33607 } }
I20260806 01:00:50.796180 9992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:00:50.796368 9992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:00:50.796552 9992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:50.798683 9992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d4ffc00b6f4e04ad2572110f75457e" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33607 } }
I20260806 01:00:50.799156 9992 leader_election.cc:302] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91d4ffc00b6f4e04ad2572110f75457e; no voters:
I20260806 01:00:50.799834 9992 leader_election.cc:288] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:00:50.800001 10002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:50.800671 10002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [term 1 LEADER]: Becoming Leader. State: Replica: 91d4ffc00b6f4e04ad2572110f75457e, State: Running, Role: LEADER
I20260806 01:00:50.801236 10002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d4ffc00b6f4e04ad2572110f75457e" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33607 } }
I20260806 01:00:50.803203 10002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91d4ffc00b6f4e04ad2572110f75457e. Latest consensus state: current_term: 1 leader_uuid: "91d4ffc00b6f4e04ad2572110f75457e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d4ffc00b6f4e04ad2572110f75457e" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33607 } } }
I20260806 01:00:50.803522 10002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:50.804138 10004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91d4ffc00b6f4e04ad2572110f75457e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d4ffc00b6f4e04ad2572110f75457e" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33607 } } }
I20260806 01:00:50.804493 10004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:50.805356 9992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:00:50.808624 10007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:00:50.814100 10007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:00:50.833159 10007 catalog_manager.cc:1347] Generated new cluster ID: 3f2ec541fe314a74aca47bf226fecd1e
I20260806 01:00:50.833313 10007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:00:50.853695 10007 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:00:50.854965 10007 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:00:50.858624 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:50.849597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.5.108.190:33607)
0806 01:00:50.849786 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:50.849792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:50.850334 (+ 542us) server_negotiation.cc:407] Connection header received
0806 01:00:50.850521 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:50.850527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:50.850622 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:50.850740 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:50.851971 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:50.852967 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:50.854820 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:50.855469 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:50.855607 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:50.856880 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:50.856887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:50.856891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:00:50.857817 (+ 926us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:50.857892 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:00:50.857923 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:50.858017 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:50.858187 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 01:00:50.858304 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:00:50.865000 10007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91d4ffc00b6f4e04ad2572110f75457e: Generated new TSK 0
I20260806 01:00:50.865900 10007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:00:50.878757 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 9880
I20260806 01:00:50.880064 9986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:50.970453 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 9880
2026-08-06T01:00:50Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2093 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:00:51 dist-test-slave-x4qh krb5kdc[10038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:00:51 dist-test-slave-x4qh krb5kdc[10038](info): set up 2 sockets
Aug 06 01:00:51 dist-test-slave-x4qh krb5kdc[10038](info): commencing operation
krb5kdc: starting...
W20260806 01:00:52.206306 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:00:52 dist-test-slave-x4qh krb5kdc[10038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:00:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:00:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:00:52.403147 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:33541
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:33541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:52.717182 10060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:52.717442 10060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:52.717623 10060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:52.724611 10060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:00:52.724737 10060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:52.724849 10060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:52.724949 10060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:00:52.725024 10060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:00:52.734521 10060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:33541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:33541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:52.735739 10060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:52.736953 10060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:52.749009 10065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:52.749073 10071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:52.749104 10067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:52.749776 10060 server_base.cc:1034] running on GCE node
I20260806 01:00:52.750502 10060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:52.751857 10060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:52.753088 10060 hybrid_clock.cc:648] HybridClock initialized: now 1785978052753034 us; error 40 us; skew 500 ppm
I20260806 01:00:52.753427 10060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:52 dist-test-slave-x4qh krb5kdc[10038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978052, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:52.756469 10060 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:00:52.757741 10060 webserver.cc:460] Webserver started at http://127.5.108.190:44951/ using document root <none> and password file <none>
I20260806 01:00:52.758328 10060 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:52.758430 10060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:52.758703 10060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:52.761682 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "298284000ba7445d974548acc982c999"
format_stamp: "Formatted at 2026-08-06 01:00:52 on dist-test-slave-x4qh"
server_key: "10229e265d12c07a411155b5232a024c"
server_key_iv: "22eacbcfeea743c0d98a76c95241c32d"
server_key_version: "encryptionkey@0"
I20260806 01:00:52.762444 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "298284000ba7445d974548acc982c999"
format_stamp: "Formatted at 2026-08-06 01:00:52 on dist-test-slave-x4qh"
server_key: "10229e265d12c07a411155b5232a024c"
server_key_iv: "22eacbcfeea743c0d98a76c95241c32d"
server_key_version: "encryptionkey@0"
I20260806 01:00:52.767443 10060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:00:52.770522 10079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:52.771723 10060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260806 01:00:52.771906 10060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "298284000ba7445d974548acc982c999"
format_stamp: "Formatted at 2026-08-06 01:00:52 on dist-test-slave-x4qh"
server_key: "10229e265d12c07a411155b5232a024c"
server_key_iv: "22eacbcfeea743c0d98a76c95241c32d"
server_key_version: "encryptionkey@0"
I20260806 01:00:52.772086 10060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:52.788511 10060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:52.792223 10060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:52.792518 10060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:52.810781 10060 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:33541
I20260806 01:00:52.810840 10157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:33541 every 8 connection(s)
I20260806 01:00:52.812050 10060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:00:52.815492 10158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:52.820052 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 10060
I20260806 01:00:52.820286 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:00:52.821012 5554 external_mini_cluster.cc:1428] Setting key 3a08b40c7738ea506b3b7f9f09002866
I20260806 01:00:52.826156 10158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999: Bootstrap starting.
I20260806 01:00:52.829438 10158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:52.830549 10158 log.cc:824] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999: Log is configured to *not* fsync() on all Append() calls
I20260806 01:00:52.833717 10158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999: No bootstrap required, opened a new log
Aug 06 01:00:52 dist-test-slave-x4qh krb5kdc[10038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:52.841311 10158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298284000ba7445d974548acc982c999" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33541 } }
I20260806 01:00:52.841655 10158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:52.841771 10158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 298284000ba7445d974548acc982c999, State: Initialized, Role: FOLLOWER
I20260806 01:00:52.842402 10158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298284000ba7445d974548acc982c999" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33541 } }
I20260806 01:00:52.842664 10158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:00:52.842816 10158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:00:52.843012 10158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:52.844125 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:52.822989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42114 (local address 127.5.108.190:33541)
0806 01:00:52.823864 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:52.823874 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:52.823916 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:00:52.825527 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:52.825547 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:52.825925 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:52.826201 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:52.827900 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:52.829014 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:52.830697 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:52.831711 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:52.831918 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:52.836581 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:52.836589 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:52.836601 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:52.836639 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:52.839810 (+ 3171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:52.840518 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:52.840529 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:52.840540 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:52.840718 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:52.841116 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:52.841121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:52.841123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:52.841595 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:52.841941 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:52.842360 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 01:00:52.842758 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":441,"threads_started":1}
I20260806 01:00:52.845983 10158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298284000ba7445d974548acc982c999" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33541 } }
I20260806 01:00:52.846549 10158 leader_election.cc:302] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 298284000ba7445d974548acc982c999; no voters:
I20260806 01:00:52.847271 10158 leader_election.cc:288] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:00:52.847429 10167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:52.848094 10167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [term 1 LEADER]: Becoming Leader. State: Replica: 298284000ba7445d974548acc982c999, State: Running, Role: LEADER
I20260806 01:00:52.848654 10167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298284000ba7445d974548acc982c999" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33541 } }
I20260806 01:00:52.849421 10158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:00:52.851621 10171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 298284000ba7445d974548acc982c999. Latest consensus state: current_term: 1 leader_uuid: "298284000ba7445d974548acc982c999" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298284000ba7445d974548acc982c999" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33541 } } }
I20260806 01:00:52.852029 10171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:52.852574 10168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "298284000ba7445d974548acc982c999" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298284000ba7445d974548acc982c999" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33541 } } }
I20260806 01:00:52.852972 10168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999 [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:52.855830 10178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:00:52.860347 10178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:00:52.868772 10178 catalog_manager.cc:1347] Generated new cluster ID: c2874039cd244fcd985b85f46a693917
I20260806 01:00:52.869079 10178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:00:52.900435 10178 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:00:52.902077 10178 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:00:52.911060 10178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 298284000ba7445d974548acc982c999: Generated new TSK 0
I20260806 01:00:52.911862 10178 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:00:52.961315 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:33541
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:53.276844 10201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:53.277101 10201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:53.277254 10201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:53.284076 10201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:53.284206 10201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:53.284374 10201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:00:53.293443 10201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:33541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:53.294571 10201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:53.295992 10201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:53.308423 10209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:53.308682 10211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:53.308693 10213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:53.309628 10201 server_base.cc:1034] running on GCE node
I20260806 01:00:53.310277 10201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:53.311187 10201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:53.312428 10201 hybrid_clock.cc:648] HybridClock initialized: now 1785978053312324 us; error 93 us; skew 500 ppm
I20260806 01:00:53.312742 10201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:53 dist-test-slave-x4qh krb5kdc[10038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978053, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:53.316246 10201 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:00:53.317687 10201 webserver.cc:460] Webserver started at http://127.5.108.129:36251/ using document root <none> and password file <none>
I20260806 01:00:53.318313 10201 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:53.318445 10201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:53.318710 10201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:53.321753 10201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "46d66cc868d7448a8169995b3056482c"
format_stamp: "Formatted at 2026-08-06 01:00:53 on dist-test-slave-x4qh"
server_key: "85130faa93f658f1e4e0c46d26762434"
server_key_iv: "8895b1907726090ea1b6cb7f9f6feb44"
server_key_version: "encryptionkey@0"
I20260806 01:00:53.322465 10201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "46d66cc868d7448a8169995b3056482c"
format_stamp: "Formatted at 2026-08-06 01:00:53 on dist-test-slave-x4qh"
server_key: "85130faa93f658f1e4e0c46d26762434"
server_key_iv: "8895b1907726090ea1b6cb7f9f6feb44"
server_key_version: "encryptionkey@0"
I20260806 01:00:53.328017 10201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:00:53.331184 10230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:53.332525 10201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:00:53.332690 10201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "46d66cc868d7448a8169995b3056482c"
format_stamp: "Formatted at 2026-08-06 01:00:53 on dist-test-slave-x4qh"
server_key: "85130faa93f658f1e4e0c46d26762434"
server_key_iv: "8895b1907726090ea1b6cb7f9f6feb44"
server_key_version: "encryptionkey@0"
I20260806 01:00:53.332909 10201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:53.360323 10201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:53.364614 10201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:53.364928 10201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:53.365861 10201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:53.367134 10201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:53.367208 10201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:53.367372 10201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:53.367458 10201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 01:00:53.400514 10201 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:35065
I20260806 01:00:53.400601 10391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:35065 every 8 connection(s)
I20260806 01:00:53.401795 10201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:00:53.409121 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 10201
I20260806 01:00:53.409322 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:00:53.409941 5554 external_mini_cluster.cc:1428] Setting key af392580b9dc72dbcecaee470c5c0e1e
Aug 06 01:00:53 dist-test-slave-x4qh krb5kdc[10038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978053, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:53.421072 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:53.404273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:55917 (local address 127.5.108.190:33541)
0806 01:00:53.404733 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:53.404738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:53.405547 (+ 809us) server_negotiation.cc:407] Connection header received
0806 01:00:53.407406 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:53.407412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:53.407512 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:53.407634 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:53.409024 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:53.409946 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:53.411756 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:53.412140 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:53.412287 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:53.415771 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:53.415776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:53.415779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:53.415807 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:53.418253 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:53.418940 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:53.418949 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:53.418951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:53.419064 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:53.419492 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:53.419496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:53.419499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:53.419748 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:53.419898 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:53.420509 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 01:00:53.420720 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":253,"threads_started":1}
I20260806 01:00:53.421905 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:53.404728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:33541 (local address 127.5.108.129:55917)
0806 01:00:53.405389 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:53.405433 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:53.407094 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:53.407862 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:53.407870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:53.408340 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:53.408847 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:53.408863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:53.410091 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:53.410100 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:53.411611 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:53.411623 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:53.412380 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:53.412387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:53.412578 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:53.413321 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:53.413354 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:53.415543 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:53.418401 (+ 2858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:53.418409 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:53.418422 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:53.418812 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:53.419171 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:53.419175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:53.419178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:53.419347 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:53.419873 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:53.419879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:53.420156 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 01:00:53.420384 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 01:00:53.420864 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":266,"threads_started":1}
I20260806 01:00:53.423295 10394 heartbeater.cc:347] Connected to a master server at 127.5.108.190:33541
I20260806 01:00:53.423616 10394 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:53.424472 10394 heartbeater.cc:511] Master 127.5.108.190:33541 requested a full tablet report, sending...
I20260806 01:00:53.426586 10107 ts_manager.cc:194] Registered new tserver with Master: 46d66cc868d7448a8169995b3056482c (127.5.108.129:35065)
I20260806 01:00:53.428015 10107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:55917
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:00:53.456148 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:33541
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:53.773865 10403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:53.774124 10403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:53.774223 10403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:53.781360 10403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:53.781456 10403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:53.781559 10403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:00:53.790560 10403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:33541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:53.791658 10403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:53.792977 10403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:53.805785 10409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:53.805737 10410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:53.805820 10412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:53.806571 10403 server_base.cc:1034] running on GCE node
I20260806 01:00:53.807161 10403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:53.808106 10403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:53.809336 10403 hybrid_clock.cc:648] HybridClock initialized: now 1785978053809291 us; error 56 us; skew 500 ppm
I20260806 01:00:53.809654 10403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:53 dist-test-slave-x4qh krb5kdc[10038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978053, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:53.812687 10403 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:00:53.823235 10403 webserver.cc:460] Webserver started at http://127.5.108.130:42765/ using document root <none> and password file <none>
I20260806 01:00:53.823915 10403 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:53.824052 10403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:53.824399 10403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:53.827123 10403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "30dd0300f1cb45bba2a30d6e2a5d566d"
format_stamp: "Formatted at 2026-08-06 01:00:53 on dist-test-slave-x4qh"
server_key: "271b0c45b959b22dac14b90497abdd14"
server_key_iv: "01df6d01ac2a1939a97307818943b301"
server_key_version: "encryptionkey@0"
I20260806 01:00:53.827931 10403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "30dd0300f1cb45bba2a30d6e2a5d566d"
format_stamp: "Formatted at 2026-08-06 01:00:53 on dist-test-slave-x4qh"
server_key: "271b0c45b959b22dac14b90497abdd14"
server_key_iv: "01df6d01ac2a1939a97307818943b301"
server_key_version: "encryptionkey@0"
I20260806 01:00:53.832582 10403 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:00:53.835988 10425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:53.837347 10403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:00:53.837505 10403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "30dd0300f1cb45bba2a30d6e2a5d566d"
format_stamp: "Formatted at 2026-08-06 01:00:53 on dist-test-slave-x4qh"
server_key: "271b0c45b959b22dac14b90497abdd14"
server_key_iv: "01df6d01ac2a1939a97307818943b301"
server_key_version: "encryptionkey@0"
I20260806 01:00:53.837693 10403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:53.868300 10403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:53.872004 10403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:53.872285 10403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:53.873245 10403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:53.874511 10403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:53.874584 10403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:53.874765 10403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:53.874817 10403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:53.908133 10403 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:45233
I20260806 01:00:53.908219 10585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:45233 every 8 connection(s)
I20260806 01:00:53.909410 10403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:00:53.914023 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 10403
I20260806 01:00:53.914233 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:00:53.914678 5554 external_mini_cluster.cc:1428] Setting key 0d31266f93739807863e932ebd81f73e
Aug 06 01:00:53 dist-test-slave-x4qh krb5kdc[10038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978053, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:53.932062 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:53.912022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:46679 (local address 127.5.108.190:33541)
0806 01:00:53.912201 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:53.912205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:53.915423 (+ 3218us) server_negotiation.cc:407] Connection header received
0806 01:00:53.917627 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:53.917632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:53.917728 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:53.917839 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:53.919349 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:53.920500 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:53.922043 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:53.922776 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:53.922925 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:53.926344 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:53.926349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:53.926352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:53.926385 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:53.928999 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:53.929706 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:53.929714 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:53.929720 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:53.929861 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:53.930293 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:53.930299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:53.930302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:53.930570 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:53.930757 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:53.931451 (+ 694us) server_negotiation.cc:299] Negotiation successful
0806 01:00:53.931710 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:00:53.933050 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:53.912542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:33541 (local address 127.5.108.130:46679)
0806 01:00:53.913448 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:53.915262 (+ 1814us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:53.917348 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:53.918051 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:53.918059 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:53.918585 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:53.919160 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:53.919177 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:53.920675 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:53.920678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:53.921896 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:53.921908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:53.922977 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:53.922982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:53.923173 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:53.923920 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:53.923948 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:53.926119 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:53.929147 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:53.929156 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:53.929171 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:53.929584 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:53.929977 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:53.929982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:53.929986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:53.930168 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:53.930718 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:53.930726 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:53.931061 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 01:00:53.931347 (+ 286us) client_negotiation.cc:240] Negotiation successful
0806 01:00:53.931892 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":414,"threads_started":1}
I20260806 01:00:53.934695 10586 heartbeater.cc:347] Connected to a master server at 127.5.108.190:33541
I20260806 01:00:53.935002 10586 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:53.935902 10586 heartbeater.cc:511] Master 127.5.108.190:33541 requested a full tablet report, sending...
I20260806 01:00:53.937505 10107 ts_manager.cc:194] Registered new tserver with Master: 30dd0300f1cb45bba2a30d6e2a5d566d (127.5.108.130:45233)
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
I20260806 01:00:53.938454 10107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:46679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:00:53.964457 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:33541
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:54.282858 10593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:54.283124 10593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:54.283309 10593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:54.290130 10593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:54.290218 10593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:54.290369 10593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:00:54.299337 10593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:33541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:54.300434 10593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:54.301746 10593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:54.314167 10600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:54.314311 10601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:54.314337 10603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:54.314960 10593 server_base.cc:1034] running on GCE node
I20260806 01:00:54.315521 10593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:54.316304 10593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:54.317524 10593 hybrid_clock.cc:648] HybridClock initialized: now 1785978054317477 us; error 35 us; skew 500 ppm
I20260806 01:00:54.317819 10593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978054, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:54.321122 10593 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:00:54.322441 10593 webserver.cc:460] Webserver started at http://127.5.108.131:37055/ using document root <none> and password file <none>
I20260806 01:00:54.323057 10593 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:54.323215 10593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:54.323572 10593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:54.326385 10593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "6eddf85ca50d4b3a9ce3570470f717b5"
format_stamp: "Formatted at 2026-08-06 01:00:54 on dist-test-slave-x4qh"
server_key: "629a45b71e20eba59f1fe632dbf6d798"
server_key_iv: "5520b25951cde60f1a6e25fa4f13fffd"
server_key_version: "encryptionkey@0"
I20260806 01:00:54.327131 10593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "6eddf85ca50d4b3a9ce3570470f717b5"
format_stamp: "Formatted at 2026-08-06 01:00:54 on dist-test-slave-x4qh"
server_key: "629a45b71e20eba59f1fe632dbf6d798"
server_key_iv: "5520b25951cde60f1a6e25fa4f13fffd"
server_key_version: "encryptionkey@0"
I20260806 01:00:54.331903 10593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:00:54.335079 10614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:54.336499 10593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:00:54.336683 10593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "6eddf85ca50d4b3a9ce3570470f717b5"
format_stamp: "Formatted at 2026-08-06 01:00:54 on dist-test-slave-x4qh"
server_key: "629a45b71e20eba59f1fe632dbf6d798"
server_key_iv: "5520b25951cde60f1a6e25fa4f13fffd"
server_key_version: "encryptionkey@0"
I20260806 01:00:54.336858 10593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:54.349988 10593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:54.354152 10593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:54.354449 10593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:54.355260 10593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:54.356477 10593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:54.356577 10593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:54.356716 10593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:54.356791 10593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:54.389717 10593 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:43763
I20260806 01:00:54.389786 10774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:43763 every 8 connection(s)
I20260806 01:00:54.390893 10593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:00:54.392584 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 10593
I20260806 01:00:54.392769 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:00:54.393306 5554 external_mini_cluster.cc:1428] Setting key 48b06f9d340ac18fb535cc18f1dcfdb2
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978054, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:54.411062 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:54.393310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:55451 (local address 127.5.108.190:33541)
0806 01:00:54.393507 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:54.393513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:54.394838 (+ 1325us) server_negotiation.cc:407] Connection header received
0806 01:00:54.396550 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:54.396554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:54.396638 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:54.396717 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:54.397997 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.398713 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.400619 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.401145 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.401251 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:54.405099 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:54.405104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:54.405107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:54.405137 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:54.408124 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.408829 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.408834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.408838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.408970 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.409424 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.409428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.409430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.409691 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:54.409865 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:54.410499 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 01:00:54.410704 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:00:54.411875 10780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:54.393805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:33541 (local address 127.5.108.131:55451)
0806 01:00:54.394638 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:54.394707 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:54.396314 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:54.396897 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:54.396904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:54.397331 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:54.397841 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:54.397857 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.398836 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.398839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:54.400464 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:54.400479 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.401367 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.401373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:54.401623 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:54.402306 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:54.402337 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:54.404883 (+ 2546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:54.408286 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:54.408295 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:54.408308 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:54.408693 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:54.409086 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:54.409091 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:54.409095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:54.409292 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:54.409840 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:54.409847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:54.410149 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 01:00:54.410389 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 01:00:54.410831 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":386,"threads_started":1}
I20260806 01:00:54.413106 10775 heartbeater.cc:347] Connected to a master server at 127.5.108.190:33541
I20260806 01:00:54.413414 10775 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:54.414086 10775 heartbeater.cc:511] Master 127.5.108.190:33541 requested a full tablet report, sending...
I20260806 01:00:54.415519 10107 ts_manager.cc:194] Registered new tserver with Master: 6eddf85ca50d4b3a9ce3570470f717b5 (127.5.108.131:43763)
I20260806 01:00:54.416313 10107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:55451
I20260806 01:00:54.420274 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:00:54.431433 10394 heartbeater.cc:503] Master 127.5.108.190:33541 was elected leader, sending a full tablet report...
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:54.471055 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:54.458045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42118 (local address 127.5.108.190:33541)
0806 01:00:54.458243 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:54.458252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:54.458769 (+ 517us) server_negotiation.cc:407] Connection header received
0806 01:00:54.458985 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:54.458992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:54.459054 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:54.459152 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:54.460559 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.461229 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.462359 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.462758 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.462883 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:54.466454 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:54.466459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:54.466461 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:54.466488 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:54.468709 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.469295 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.469302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.469309 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.469424 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.469775 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.469780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.469783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.470060 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:54.470301 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:54.470562 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 01:00:54.470761 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 01:00:54.471676 10107 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42118
I20260806 01:00:54.473389 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 10201
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:00:54.488524 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:54.474371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54934 (local address 127.5.108.129:35065)
0806 01:00:54.474947 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:54.474954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:54.474976 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:00:54.475024 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:54.475029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:54.475229 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:54.475443 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:54.476601 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.477501 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.478756 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.479298 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.479488 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:54.483113 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:54.483124 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:54.483136 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:54.483170 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:54.485799 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.486447 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.486454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.486456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.486581 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.486958 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.486963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.486967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.487289 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:54.487637 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:54.487932 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 01:00:54.488181 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":285,"threads_started":1}
W20260806 01:00:54.489606 10387 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54934
W20260806 01:00:54.490525 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 10201: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:00:54.490712 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 10201
I20260806 01:00:54.518747 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 10403
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:00:54.533375 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:54.519653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32868 (local address 127.5.108.130:45233)
0806 01:00:54.520185 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:54.520192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:54.520215 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:00:54.520280 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:54.520285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:54.520476 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:54.520596 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:54.521694 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.522677 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.523905 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.524449 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.524544 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:54.528364 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:54.528374 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:54.528386 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:54.528417 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:54.530889 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.531534 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.531540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.531542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.531661 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.531972 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.531975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.531976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.532336 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:54.532553 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:54.532780 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 01:00:54.532966 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":277,"threads_started":1}
W20260806 01:00:54.534515 10579 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32868
W20260806 01:00:54.535508 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 10403: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:00:54.535634 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 10403
I20260806 01:00:54.558526 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 10593
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:00:54.573577 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:54.559582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.5.108.131:43763)
0806 01:00:54.560202 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:54.560209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:54.560235 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:00:54.560291 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:54.560297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:54.560472 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:54.560597 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:54.561788 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.562707 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.563969 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:54.564528 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:54.564624 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:54.568675 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:54.568685 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:54.568697 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:54.568728 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:54.571175 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.571918 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.571924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.571926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.572039 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:54.572365 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:54.572368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:54.572370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:54.572660 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:54.572882 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:54.573102 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 01:00:54.573296 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":321,"threads_started":1}
W20260806 01:00:54.574780 10765 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38648
W20260806 01:00:54.575860 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 10593: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:00:54.576011 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 10593
I20260806 01:00:54.600178 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 10060
I20260806 01:00:54.601320 10150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:54.702657 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 10060
2026-08-06T01:00:54Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3737 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10799](info): set up 2 sockets
Aug 06 01:00:54 dist-test-slave-x4qh krb5kdc[10799](info): commencing operation
krb5kdc: starting...
W20260806 01:00:55.962337 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:00:56 dist-test-slave-x4qh krb5kdc[10799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:00:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:00:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:00:56.158181 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44789
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:44789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:56.465567 10817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:56.465860 10817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:56.466012 10817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:56.472579 10817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:00:56.472718 10817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:56.472811 10817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:56.472901 10817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:00:56.472975 10817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:00:56.481485 10817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:44789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:56.482635 10817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:56.483944 10817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:56.495014 10824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:56.494980 10829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:56.495038 10826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:56.496218 10817 server_base.cc:1034] running on GCE node
I20260806 01:00:56.496918 10817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:56.498072 10817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:56.499300 10817 hybrid_clock.cc:648] HybridClock initialized: now 1785978056499260 us; error 48 us; skew 500 ppm
I20260806 01:00:56.499646 10817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:56 dist-test-slave-x4qh krb5kdc[10799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978056, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:56.502579 10817 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:00:56.504151 10817 webserver.cc:460] Webserver started at http://127.5.108.190:36569/ using document root <none> and password file <none>
I20260806 01:00:56.504840 10817 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:56.504987 10817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:56.505326 10817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:56.508391 10817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "5fb23cbd646941188f145ee4108fd72b"
format_stamp: "Formatted at 2026-08-06 01:00:56 on dist-test-slave-x4qh"
server_key: "6b5711315eb53a7a77f2c734f64da0ce"
server_key_iv: "48e1e56419c051a56f7cc692edd04d7a"
server_key_version: "encryptionkey@0"
I20260806 01:00:56.509203 10817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "5fb23cbd646941188f145ee4108fd72b"
format_stamp: "Formatted at 2026-08-06 01:00:56 on dist-test-slave-x4qh"
server_key: "6b5711315eb53a7a77f2c734f64da0ce"
server_key_iv: "48e1e56419c051a56f7cc692edd04d7a"
server_key_version: "encryptionkey@0"
I20260806 01:00:56.514047 10817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:00:56.517279 10839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:56.518406 10817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 01:00:56.518572 10817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "5fb23cbd646941188f145ee4108fd72b"
format_stamp: "Formatted at 2026-08-06 01:00:56 on dist-test-slave-x4qh"
server_key: "6b5711315eb53a7a77f2c734f64da0ce"
server_key_iv: "48e1e56419c051a56f7cc692edd04d7a"
server_key_version: "encryptionkey@0"
I20260806 01:00:56.518764 10817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:56.534626 10817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:56.538702 10817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:56.539014 10817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:56.557258 10817 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:44789
I20260806 01:00:56.557303 10914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:44789 every 8 connection(s)
I20260806 01:00:56.558502 10817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:00:56.561911 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:00:56.565438 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 10817
I20260806 01:00:56.565699 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:00:56.566555 5554 external_mini_cluster.cc:1428] Setting key 417d3b1b749f10505dd8ed1edc678ae4
I20260806 01:00:56.575073 10915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b: Bootstrap starting.
I20260806 01:00:56.578728 10915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b: Neither blocks nor log segments found. Creating new log.
I20260806 01:00:56.580104 10915 log.cc:824] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:00:56 dist-test-slave-x4qh krb5kdc[10799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:56.582912 10915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b: No bootstrap required, opened a new log
I20260806 01:00:56.589638 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:56.568566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.5.108.190:44789)
0806 01:00:56.569331 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:56.569342 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:56.569381 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:00:56.570971 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:56.570990 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:56.571351 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:56.571727 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:56.573500 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:56.574791 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:56.576657 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:56.577318 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:56.577445 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:56.582256 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:56.582263 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:56.582276 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:56.582308 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:56.585599 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:56.586335 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:56.586350 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:56.586359 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:56.586545 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:56.586955 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:56.586966 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:56.586969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:56.587545 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:56.587871 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:56.588324 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 01:00:56.588711 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":385,"threads_started":1}
I20260806 01:00:56.590690 10915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb23cbd646941188f145ee4108fd72b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44789 } }
I20260806 01:00:56.591075 10915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:00:56.591178 10915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb23cbd646941188f145ee4108fd72b, State: Initialized, Role: FOLLOWER
I20260806 01:00:56.591758 10915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb23cbd646941188f145ee4108fd72b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44789 } }
I20260806 01:00:56.592147 10915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:00:56.592340 10915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:00:56.592602 10915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:00:56.595304 10915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb23cbd646941188f145ee4108fd72b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44789 } }
I20260806 01:00:56.595831 10915 leader_election.cc:302] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fb23cbd646941188f145ee4108fd72b; no voters:
I20260806 01:00:56.596386 10915 leader_election.cc:288] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:00:56.596504 10922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:00:56.597488 10922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [term 1 LEADER]: Becoming Leader. State: Replica: 5fb23cbd646941188f145ee4108fd72b, State: Running, Role: LEADER
I20260806 01:00:56.598237 10922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb23cbd646941188f145ee4108fd72b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44789 } }
I20260806 01:00:56.598418 10915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:00:56.600780 10928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fb23cbd646941188f145ee4108fd72b. Latest consensus state: current_term: 1 leader_uuid: "5fb23cbd646941188f145ee4108fd72b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb23cbd646941188f145ee4108fd72b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44789 } } }
I20260806 01:00:56.601210 10928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:56.601843 10927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fb23cbd646941188f145ee4108fd72b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb23cbd646941188f145ee4108fd72b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44789 } } }
I20260806 01:00:56.602185 10927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b [sys.catalog]: This master's current role is: LEADER
I20260806 01:00:56.602355 10940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:00:56.611235 10940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:00:56.626529 10940 catalog_manager.cc:1347] Generated new cluster ID: 803c59fca14348fc86a616aa384c9241
I20260806 01:00:56.626667 10940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:00:56.641860 10940 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:00:56.643654 10940 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:00:56.655133 10940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fb23cbd646941188f145ee4108fd72b: Generated new TSK 0
I20260806 01:00:56.655977 10940 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:00:56.710273 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44789
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:57.028559 10957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:57.028846 10957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:57.029021 10957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:57.035821 10957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:57.035954 10957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:57.036134 10957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:00:57.045270 10957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:57.046406 10957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:57.047708 10957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:57.060477 10962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:57.060590 10963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:57.060567 10965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:57.061162 10957 server_base.cc:1034] running on GCE node
I20260806 01:00:57.061939 10957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:57.062853 10957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:57.064082 10957 hybrid_clock.cc:648] HybridClock initialized: now 1785978057064036 us; error 58 us; skew 500 ppm
I20260806 01:00:57.064409 10957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:57 dist-test-slave-x4qh krb5kdc[10799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978057, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:57.067415 10957 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:00:57.068801 10957 webserver.cc:460] Webserver started at http://127.5.108.129:37373/ using document root <none> and password file <none>
I20260806 01:00:57.069525 10957 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:57.069674 10957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:57.069983 10957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:57.072752 10957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "5e169a10da484bc5b4980b9fe5acd7ff"
format_stamp: "Formatted at 2026-08-06 01:00:57 on dist-test-slave-x4qh"
server_key: "458745b31dc2060e5e7c3e5588d79a30"
server_key_iv: "f8995bc020300f976073f054255f0409"
server_key_version: "encryptionkey@0"
I20260806 01:00:57.073550 10957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "5e169a10da484bc5b4980b9fe5acd7ff"
format_stamp: "Formatted at 2026-08-06 01:00:57 on dist-test-slave-x4qh"
server_key: "458745b31dc2060e5e7c3e5588d79a30"
server_key_iv: "f8995bc020300f976073f054255f0409"
server_key_version: "encryptionkey@0"
I20260806 01:00:57.078311 10957 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 01:00:57.081578 10980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:57.083056 10957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:00:57.083245 10957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "5e169a10da484bc5b4980b9fe5acd7ff"
format_stamp: "Formatted at 2026-08-06 01:00:57 on dist-test-slave-x4qh"
server_key: "458745b31dc2060e5e7c3e5588d79a30"
server_key_iv: "f8995bc020300f976073f054255f0409"
server_key_version: "encryptionkey@0"
I20260806 01:00:57.083513 10957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:57.101802 10957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:57.105741 10957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:57.106033 10957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:57.106947 10957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:57.108294 10957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:57.108366 10957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:57.108534 10957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:57.108608 10957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:57.141306 10957 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:45623
I20260806 01:00:57.141394 11140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:45623 every 8 connection(s)
I20260806 01:00:57.142593 10957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:00:57.150609 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 10957
I20260806 01:00:57.150882 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:00:57.151585 5554 external_mini_cluster.cc:1428] Setting key 6fad6f9937e82c247456147fa2fdb01a
Aug 06 01:00:57 dist-test-slave-x4qh krb5kdc[10799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978057, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:57.163010 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:57.145263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:60609 (local address 127.5.108.190:44789)
0806 01:00:57.145729 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:57.145737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:57.146586 (+ 849us) server_negotiation.cc:407] Connection header received
0806 01:00:57.148372 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:57.148376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:57.148442 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:57.148537 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:57.150010 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:57.150750 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:57.152446 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:57.152883 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:57.153017 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:57.156928 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:57.156933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:57.156936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:57.156965 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:57.159509 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:57.160441 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:57.160445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:57.160447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:57.160566 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:57.161049 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:57.161052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:57.161054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:57.161316 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:57.161423 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:57.162308 (+ 885us) server_negotiation.cc:299] Negotiation successful
0806 01:00:57.162565 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
I20260806 01:00:57.163563 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:57.145778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44789 (local address 127.5.108.129:60609)
0806 01:00:57.146434 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:57.146474 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:57.148138 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:57.148738 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:57.148745 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:57.149241 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:57.149843 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:57.149862 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:57.150939 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:57.150942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:57.152285 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:57.152298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:57.153101 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:57.153112 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:57.153387 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:57.154050 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:57.154072 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:57.156712 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:57.159631 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:57.159641 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:57.159655 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:57.160311 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:57.160673 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:57.160680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:57.160686 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:57.160928 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:57.161437 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:57.161447 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:57.161855 (+ 408us) client_negotiation.cc:769] Sending connection context
0806 01:00:57.162160 (+ 305us) client_negotiation.cc:240] Negotiation successful
0806 01:00:57.162492 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":245,"threads_started":1}
I20260806 01:00:57.164949 11142 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44789
I20260806 01:00:57.165231 11142 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:57.165899 11142 heartbeater.cc:511] Master 127.5.108.190:44789 requested a full tablet report, sending...
I20260806 01:00:57.167907 10868 ts_manager.cc:194] Registered new tserver with Master: 5e169a10da484bc5b4980b9fe5acd7ff (127.5.108.129:45623)
I20260806 01:00:57.169133 10868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:60609
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:00:57.198385 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44789
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:57.517745 11148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:57.518038 11148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:57.518217 11148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:57.525274 11148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:57.525415 11148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:57.525579 11148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:00:57.534549 11148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:57.535656 11148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:57.536806 11148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:57.549437 11159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:57.549459 11166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:57.549408 11160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:57.550153 11148 server_base.cc:1034] running on GCE node
I20260806 01:00:57.550665 11148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:57.551575 11148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:57.552776 11148 hybrid_clock.cc:648] HybridClock initialized: now 1785978057552715 us; error 52 us; skew 500 ppm
I20260806 01:00:57.553047 11148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:57 dist-test-slave-x4qh krb5kdc[10799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978057, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:57.555966 11148 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:00:57.557585 11148 webserver.cc:460] Webserver started at http://127.5.108.130:41169/ using document root <none> and password file <none>
I20260806 01:00:57.558355 11148 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:57.558487 11148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:57.558804 11148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:57.562376 11148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "9dc3aa9802ff4c759a258a6da7556e54"
format_stamp: "Formatted at 2026-08-06 01:00:57 on dist-test-slave-x4qh"
server_key: "f9fa52178b05e1163b0d011d7a1f4981"
server_key_iv: "4e2e4a260a37ee00745eb23c75d539ae"
server_key_version: "encryptionkey@0"
I20260806 01:00:57.563364 11148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "9dc3aa9802ff4c759a258a6da7556e54"
format_stamp: "Formatted at 2026-08-06 01:00:57 on dist-test-slave-x4qh"
server_key: "f9fa52178b05e1163b0d011d7a1f4981"
server_key_iv: "4e2e4a260a37ee00745eb23c75d539ae"
server_key_version: "encryptionkey@0"
I20260806 01:00:57.569377 11148 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:00:57.573406 11174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:57.574851 11148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:00:57.574997 11148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "9dc3aa9802ff4c759a258a6da7556e54"
format_stamp: "Formatted at 2026-08-06 01:00:57 on dist-test-slave-x4qh"
server_key: "f9fa52178b05e1163b0d011d7a1f4981"
server_key_iv: "4e2e4a260a37ee00745eb23c75d539ae"
server_key_version: "encryptionkey@0"
I20260806 01:00:57.575215 11148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:57.595917 11148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:57.599539 11148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:57.599763 11148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:57.600629 11148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:57.601886 11148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:57.601989 11148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:57.602097 11148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:57.602149 11148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:57.633960 11148 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:33025
I20260806 01:00:57.634044 11343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:33025 every 8 connection(s)
I20260806 01:00:57.635279 11148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:00:57.645467 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 11148
I20260806 01:00:57.645709 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:00:57.646313 5554 external_mini_cluster.cc:1428] Setting key d3d0783da12fcb3c11272b37503563ab
Aug 06 01:00:57 dist-test-slave-x4qh krb5kdc[10799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978057, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:57.655413 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:57.637755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:35965 (local address 127.5.108.190:44789)
0806 01:00:57.637959 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:57.637965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:57.639046 (+ 1081us) server_negotiation.cc:407] Connection header received
0806 01:00:57.640696 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:57.640699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:57.640764 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:57.640846 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:57.642118 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:57.642833 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:57.644483 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:57.644875 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:57.644969 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:57.648875 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:57.648881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:57.648886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:57.648922 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:57.652415 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:57.653028 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:57.653036 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:57.653042 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:57.653194 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:57.653605 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:57.653613 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:57.653619 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:57.654000 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:57.654157 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:57.654725 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 01:00:57.654964 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:00:57.656019 11347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:57.638204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44789 (local address 127.5.108.130:35965)
0806 01:00:57.638899 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:57.638940 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:57.640482 (+ 1542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:57.641024 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:57.641030 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:57.641495 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:57.641964 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:57.641978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:57.642982 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:57.642986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:57.644358 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:57.644370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:57.645054 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:57.645058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:57.645382 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:57.646094 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:57.646137 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:57.648663 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:57.652535 (+ 3872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:57.652541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:57.652550 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:57.652911 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:57.653319 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:57.653322 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:57.653324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:57.653488 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:57.654119 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:57.654126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:57.654395 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 01:00:57.654598 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 01:00:57.654901 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":286,"threads_started":1}
I20260806 01:00:57.657212 11344 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44789
I20260806 01:00:57.657480 11344 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:57.658135 11344 heartbeater.cc:511] Master 127.5.108.190:44789 requested a full tablet report, sending...
I20260806 01:00:57.659518 10868 ts_manager.cc:194] Registered new tserver with Master: 9dc3aa9802ff4c759a258a6da7556e54 (127.5.108.130:33025)
I20260806 01:00:57.660210 10868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:35965
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:00:57.693723 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44789
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:00:58.015340 11351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:00:58.015691 11351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:00:58.015851 11351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:00:58.022758 11351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:00:58.022879 11351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:00:58.023056 11351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:00:58.031924 11351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:00:58.033005 11351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:00:58.034264 11351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:00:58.046802 11357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:58.046875 11360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:00:58.046836 11362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:00:58.049094 11351 server_base.cc:1034] running on GCE node
I20260806 01:00:58.049692 11351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:00:58.050558 11351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:00:58.051779 11351 hybrid_clock.cc:648] HybridClock initialized: now 1785978058051726 us; error 43 us; skew 500 ppm
I20260806 01:00:58.052057 11351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[10799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978058, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:00:58.054978 11351 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:00:58.056265 11351 webserver.cc:460] Webserver started at http://127.5.108.131:35925/ using document root <none> and password file <none>
I20260806 01:00:58.056870 11351 fs_manager.cc:359] Metadata directory not provided
I20260806 01:00:58.056965 11351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:00:58.057190 11351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:00:58.059842 11351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "0cdc8a5681e74f8d8691f874d30af77d"
format_stamp: "Formatted at 2026-08-06 01:00:58 on dist-test-slave-x4qh"
server_key: "92398de482ab3aa6fba2f17f2f55b1ff"
server_key_iv: "9a9b512598bd9a38a8e27535eb87354c"
server_key_version: "encryptionkey@0"
I20260806 01:00:58.060495 11351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "0cdc8a5681e74f8d8691f874d30af77d"
format_stamp: "Formatted at 2026-08-06 01:00:58 on dist-test-slave-x4qh"
server_key: "92398de482ab3aa6fba2f17f2f55b1ff"
server_key_iv: "9a9b512598bd9a38a8e27535eb87354c"
server_key_version: "encryptionkey@0"
I20260806 01:00:58.065088 11351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:00:58.068094 11373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:58.069197 11351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 01:00:58.069352 11351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "0cdc8a5681e74f8d8691f874d30af77d"
format_stamp: "Formatted at 2026-08-06 01:00:58 on dist-test-slave-x4qh"
server_key: "92398de482ab3aa6fba2f17f2f55b1ff"
server_key_iv: "9a9b512598bd9a38a8e27535eb87354c"
server_key_version: "encryptionkey@0"
I20260806 01:00:58.069567 11351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:00:58.083273 11351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:00:58.086829 11351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:00:58.087078 11351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:00:58.088066 11351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:00:58.089279 11351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:00:58.089386 11351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:58.089524 11351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:00:58.089591 11351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:00:58.121285 11351 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:38741
I20260806 01:00:58.121356 11529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:38741 every 8 connection(s)
I20260806 01:00:58.122529 11351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:00:58.130844 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 11351
I20260806 01:00:58.131031 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:00:58.131508 5554 external_mini_cluster.cc:1428] Setting key b813a7cea881108cd188db55057f9bd5
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[10799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978058, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:58.141841 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:58.124948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:43477 (local address 127.5.108.190:44789)
0806 01:00:58.125127 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:58.125131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:58.126231 (+ 1100us) server_negotiation.cc:407] Connection header received
0806 01:00:58.128078 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:58.128083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:58.128186 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:58.128301 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:58.129668 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.130438 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.132224 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.132922 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.133047 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:58.136554 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:58.136558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:58.136561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:58.136591 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:58.139053 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.139826 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.139830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.139832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.139947 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.140308 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.140311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.140313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.140541 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:58.140646 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:58.141451 (+ 805us) server_negotiation.cc:299] Negotiation successful
0806 01:00:58.141609 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:00:58.143031 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:58.125401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44789 (local address 127.5.108.131:43477)
0806 01:00:58.126082 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 01:00:58.126117 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:00:58.127820 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:00:58.128523 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:00:58.128531 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:00:58.128984 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:00:58.129499 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:58.129515 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.130622 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.130626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:58.132042 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:00:58.132057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.133117 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.133122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:00:58.133436 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:58.134104 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:00:58.134136 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:00:58.136363 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:00:58.139240 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:58.139246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:58.139255 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:58.139719 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:58.140037 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:00:58.140040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:00:58.140042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:00:58.140211 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:00:58.140671 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:00:58.140678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:00:58.141082 (+ 404us) client_negotiation.cc:769] Sending connection context
0806 01:00:58.141350 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 01:00:58.141822 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":278,"threads_started":1}
I20260806 01:00:58.144466 11531 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44789
I20260806 01:00:58.144761 11531 heartbeater.cc:464] Registering TS with master...
I20260806 01:00:58.145641 11531 heartbeater.cc:511] Master 127.5.108.190:44789 requested a full tablet report, sending...
I20260806 01:00:58.147192 10868 ts_manager.cc:194] Registered new tserver with Master: 0cdc8a5681e74f8d8691f874d30af77d (127.5.108.131:38741)
I20260806 01:00:58.147945 10868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:43477
I20260806 01:00:58.158690 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:00:58.171706 11142 heartbeater.cc:503] Master 127.5.108.190:44789 was elected leader, sending a full tablet report...
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[10799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[10799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:00:58.210757 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:58.196271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.5.108.129:45623)
0806 01:00:58.196893 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:58.196900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:58.196926 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:00:58.196987 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:58.196994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:58.197197 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:58.197317 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:58.198633 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.199763 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.200918 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.201549 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.201670 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:58.205586 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:58.205597 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:58.205610 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:58.205642 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:58.208377 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.208939 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.208946 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.208948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.209064 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.209371 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.209374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.209375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.209707 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:58.209962 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:58.210187 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 01:00:58.210369 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":323,"threads_started":1}
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[10799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:00:58.228227 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:58.214852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.5.108.190:44789)
0806 01:00:58.215066 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:58.215071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:58.215281 (+ 210us) server_negotiation.cc:407] Connection header received
0806 01:00:58.215497 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:58.215501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:58.215573 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:58.215728 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:58.216956 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.217663 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.218847 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.219232 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.219369 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:58.223470 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:58.223475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:58.223478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:58.223508 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:58.225760 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.226340 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.226346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.226349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.226481 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.226833 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.226837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.226841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.227121 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:58.227416 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:58.227715 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 01:00:58.227932 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:00:58.228721 10868 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37952
I20260806 01:00:58.229606 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 10957
I20260806 01:00:58.240562 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:58.230366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60172 (local address 127.5.108.129:45623)
0806 01:00:58.230494 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:58.230498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:58.230816 (+ 318us) server_negotiation.cc:407] Connection header received
0806 01:00:58.230924 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:58.230927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:58.230991 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:58.231059 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:58.232144 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.232714 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.233790 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.234194 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.234287 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:58.236248 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:58.236252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:58.236255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:58.236279 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:58.238565 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.239041 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.239044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.239046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.239152 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.239489 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.239492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.239493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.239692 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:58.239846 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:58.240054 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 01:00:58.240245 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 01:00:58.241127 11135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:58.321132 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 10957
I20260806 01:00:58.346202 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 11148
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[10799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:00:58.362016 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:58.347091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.5.108.130:33025)
0806 01:00:58.347668 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:58.347676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:58.347700 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:00:58.347772 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:58.347778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:58.347991 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:58.348172 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:58.349473 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.350556 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.351909 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.352429 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.352572 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:58.356734 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:58.356750 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:58.356763 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:58.356799 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:58.359318 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.360024 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.360031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.360033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.360156 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.360545 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.360549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.360551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.360876 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:58.361170 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:58.361458 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 01:00:58.361701 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":390,"threads_started":1}
I20260806 01:00:58.363289 11337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:58.441013 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 11148
I20260806 01:00:58.465328 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 11351
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[10799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:00:58.481113 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:00:58.466393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.5.108.131:38741)
0806 01:00:58.467018 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0806 01:00:58.467024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:00:58.467051 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:00:58.467109 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:00:58.467114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:00:58.467318 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:00:58.467518 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:00:58.468681 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.469637 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.470812 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:00:58.471349 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:00:58.471592 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:00:58.475983 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:00:58.475994 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:00:58.476005 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:00:58.476035 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:00:58.478849 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.479456 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.479462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.479464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.479577 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:00:58.479865 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:00:58.479868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:00:58.479870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:00:58.480180 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:00:58.480391 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 01:00:58.480638 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 01:00:58.480817 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":326,"threads_started":1}
I20260806 01:00:58.482316 11524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:58.564468 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 11351
I20260806 01:00:58.588021 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 10817
I20260806 01:00:58.589246 10908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:00:58.690382 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 10817
2026-08-06T01:00:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3989 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[11558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[11558](info): set up 2 sockets
Aug 06 01:00:58 dist-test-slave-x4qh krb5kdc[11558](info): commencing operation
krb5kdc: starting...
W20260806 01:00:59.917224 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:01:00 dist-test-slave-x4qh krb5kdc[11558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:01:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:01:00.130319 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36445
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:00.450901 11578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:00.451231 11578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:00.451419 11578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:00.459028 11578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:00.459197 11578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:00.459280 11578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:00.459352 11578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:00.459457 11578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:00.468701 11578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:00.469782 11578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:00.470985 11578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:00.482755 11586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:00.482712 11588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:00.483050 11585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:00.483757 11578 server_base.cc:1034] running on GCE node
I20260806 01:01:00.484511 11578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:00.485818 11578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:00.487035 11578 hybrid_clock.cc:648] HybridClock initialized: now 1785978060486978 us; error 47 us; skew 500 ppm
I20260806 01:01:00.487414 11578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:00 dist-test-slave-x4qh krb5kdc[11558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978060, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:00.490371 11578 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:01:00.491575 11578 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:00.491684 11578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:00.492010 11578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:00.494804 11578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "82f09a20e85c45fa94df6d8d591f5595"
format_stamp: "Formatted at 2026-08-06 01:01:00 on dist-test-slave-x4qh"
server_key: "e3903453112b3e5cfc7d55e81c3278f1"
server_key_iv: "05418fd4f3bb5eec7d52167b099bcf70"
server_key_version: "encryptionkey@0"
I20260806 01:01:00.495676 11578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "82f09a20e85c45fa94df6d8d591f5595"
format_stamp: "Formatted at 2026-08-06 01:01:00 on dist-test-slave-x4qh"
server_key: "e3903453112b3e5cfc7d55e81c3278f1"
server_key_iv: "05418fd4f3bb5eec7d52167b099bcf70"
server_key_version: "encryptionkey@0"
I20260806 01:01:00.500592 11578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:01:00.503901 11598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:00.505157 11578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:01:00.505336 11578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "82f09a20e85c45fa94df6d8d591f5595"
format_stamp: "Formatted at 2026-08-06 01:01:00 on dist-test-slave-x4qh"
server_key: "e3903453112b3e5cfc7d55e81c3278f1"
server_key_iv: "05418fd4f3bb5eec7d52167b099bcf70"
server_key_version: "encryptionkey@0"
I20260806 01:01:00.505545 11578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:00.517535 11578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:00.521123 11578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:00.521385 11578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:00.539458 11578 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36445
I20260806 01:01:00.539477 11675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36445 every 8 connection(s)
I20260806 01:01:00.540280 11578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:00.543684 11676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:00.548637 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 11578
I20260806 01:01:00.548888 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:00.549503 5554 external_mini_cluster.cc:1428] Setting key c9ba1e793b011476d6577fc2361852db
I20260806 01:01:00.554982 11676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595: Bootstrap starting.
I20260806 01:01:00.558445 11676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:00.559685 11676 log.cc:824] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:00.562676 11676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595: No bootstrap required, opened a new log
Aug 06 01:01:00 dist-test-slave-x4qh krb5kdc[11558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:00.571880 11676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f09a20e85c45fa94df6d8d591f5595" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36445 } }
I20260806 01:01:00.572332 11676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:00.572460 11676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f09a20e85c45fa94df6d8d591f5595, State: Initialized, Role: FOLLOWER
I20260806 01:01:00.573078 11676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f09a20e85c45fa94df6d8d591f5595" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36445 } }
I20260806 01:01:00.573304 11676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:00.573447 11676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:00.573570 11680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:00.551686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56310 (local address 127.5.108.190:36445)
0806 01:01:00.552551 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:00.552566 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:00.552622 (+ 56us) server_negotiation.cc:407] Connection header received
0806 01:01:00.555026 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:00.555052 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:00.555528 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:00.555910 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:00.557191 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:00.559144 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:00.560669 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:00.561608 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:00.561776 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:00.566136 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:00.566144 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:00.566157 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:00.566203 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:00.569293 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:00.570094 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:00.570099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:00.570105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:00.570282 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:00.570672 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:00.570675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:00.570677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:00.571192 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:00.571469 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:00.571898 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 01:01:00.572220 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":356,"threads_started":1}
I20260806 01:01:00.573653 11676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:00.576653 11676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f09a20e85c45fa94df6d8d591f5595" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36445 } }
I20260806 01:01:00.577147 11676 leader_election.cc:302] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82f09a20e85c45fa94df6d8d591f5595; no voters:
I20260806 01:01:00.577714 11676 leader_election.cc:288] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:00.577841 11682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:00.578572 11682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [term 1 LEADER]: Becoming Leader. State: Replica: 82f09a20e85c45fa94df6d8d591f5595, State: Running, Role: LEADER
I20260806 01:01:00.579064 11682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f09a20e85c45fa94df6d8d591f5595" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36445 } }
I20260806 01:01:00.579958 11676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:00.581674 11687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82f09a20e85c45fa94df6d8d591f5595. Latest consensus state: current_term: 1 leader_uuid: "82f09a20e85c45fa94df6d8d591f5595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f09a20e85c45fa94df6d8d591f5595" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36445 } } }
I20260806 01:01:00.582067 11687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:00.582872 11685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82f09a20e85c45fa94df6d8d591f5595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f09a20e85c45fa94df6d8d591f5595" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36445 } } }
I20260806 01:01:00.583299 11685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:00.584218 11695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:00.588900 11695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:00.597218 11695 catalog_manager.cc:1347] Generated new cluster ID: f358cf3d02a1452abbdffe32ce639389
I20260806 01:01:00.597323 11695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:00.611425 11695 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:00.612520 11695 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:00.618299 11695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595: Generated new TSK 0
I20260806 01:01:00.619128 11695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:01:00.676509 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36445
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:01.000247 11718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:01.000545 11718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:01.000706 11718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:01.008018 11718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:01.008150 11718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:01.008311 11718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:01.017638 11718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:01.018831 11718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:01.020108 11718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:01.032729 11729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:01.032717 11730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:01.032799 11732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:01.033629 11718 server_base.cc:1034] running on GCE node
I20260806 01:01:01.034293 11718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:01.035113 11718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:01.036309 11718 hybrid_clock.cc:648] HybridClock initialized: now 1785978061036237 us; error 69 us; skew 500 ppm
I20260806 01:01:01.036643 11718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:01 dist-test-slave-x4qh krb5kdc[11558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978061, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:01.044111 11718 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:01:01.045239 11718 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:01.045336 11718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:01.045578 11718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:01.048525 11718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "03a739e9bed0483ab01977b661d8fbd4"
format_stamp: "Formatted at 2026-08-06 01:01:01 on dist-test-slave-x4qh"
server_key: "3bb8453e6994e10228666c234c2fcac1"
server_key_iv: "606e025ebf224a79c5f401c024c8a111"
server_key_version: "encryptionkey@0"
I20260806 01:01:01.049278 11718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "03a739e9bed0483ab01977b661d8fbd4"
format_stamp: "Formatted at 2026-08-06 01:01:01 on dist-test-slave-x4qh"
server_key: "3bb8453e6994e10228666c234c2fcac1"
server_key_iv: "606e025ebf224a79c5f401c024c8a111"
server_key_version: "encryptionkey@0"
I20260806 01:01:01.054119 11718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:01:01.057209 11740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:01.058377 11718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:01:01.058555 11718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "03a739e9bed0483ab01977b661d8fbd4"
format_stamp: "Formatted at 2026-08-06 01:01:01 on dist-test-slave-x4qh"
server_key: "3bb8453e6994e10228666c234c2fcac1"
server_key_iv: "606e025ebf224a79c5f401c024c8a111"
server_key_version: "encryptionkey@0"
I20260806 01:01:01.058780 11718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:01.102681 11718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:01.107106 11718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:01.107450 11718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:01.108340 11718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:01.109633 11718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:01.109757 11718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:01.109880 11718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:01.109962 11718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:01.142330 11718 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:40649
I20260806 01:01:01.142413 11897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:40649 every 8 connection(s)
I20260806 01:01:01.143158 11718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:01.147769 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 11718
I20260806 01:01:01.147996 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:01:01.148869 5554 external_mini_cluster.cc:1428] Setting key 11926f1443becb28024c46096605e0eb
Aug 06 01:01:01 dist-test-slave-x4qh krb5kdc[11558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978061, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:01.165359 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:01.145601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:36849 (local address 127.5.108.190:36445)
0806 01:01:01.146004 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:01.146008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:01.146952 (+ 944us) server_negotiation.cc:407] Connection header received
0806 01:01:01.149642 (+ 2690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:01.149647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:01.149750 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:01.149872 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:01.151544 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:01.152219 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:01.154677 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:01.155088 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:01.155175 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:01.159281 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:01.159286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:01.159289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:01.159320 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:01.162000 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:01.162915 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:01.162920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:01.162925 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:01.163044 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:01.163599 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:01.163606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:01.163614 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:01.163896 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:01.164053 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:01.164786 (+ 733us) server_negotiation.cc:299] Negotiation successful
0806 01:01:01.165002 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":190,"threads_started":1}
I20260806 01:01:01.166343 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:01.146107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36445 (local address 127.5.108.129:36849)
0806 01:01:01.146790 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:01.146825 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:01.149340 (+ 2515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:01.150105 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:01.150112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:01.150594 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:01.151337 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:01.151363 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:01.152389 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:01.152398 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:01.154523 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:01.154542 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:01.155334 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:01.155343 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:01.155692 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:01.156401 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:01.156436 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:01.159059 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:01.162175 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:01.162184 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:01.162205 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:01.162791 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:01.163156 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:01.163161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:01.163165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:01.163476 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:01.164038 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:01.164045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:01.164400 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 01:01:01.164667 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 01:01:01.165150 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":269,"threads_started":1}
I20260806 01:01:01.167901 11900 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36445
I20260806 01:01:01.168224 11900 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:01.169055 11900 heartbeater.cc:511] Master 127.5.108.190:36445 requested a full tablet report, sending...
I20260806 01:01:01.171546 11626 ts_manager.cc:194] Registered new tserver with Master: 03a739e9bed0483ab01977b661d8fbd4 (127.5.108.129:40649)
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
I20260806 01:01:01.173604 11626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:36849
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:01:01.200646 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36445
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:01.521049 11906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:01.521294 11906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:01.521410 11906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:01.528455 11906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:01.528573 11906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:01.528764 11906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:01.537945 11906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:01.539224 11906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:01.540486 11906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:01.553455 11914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:01.553989 11916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:01.554118 11918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:01.554365 11906 server_base.cc:1034] running on GCE node
I20260806 01:01:01.554950 11906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:01.555907 11906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:01.557130 11906 hybrid_clock.cc:648] HybridClock initialized: now 1785978061557089 us; error 47 us; skew 500 ppm
I20260806 01:01:01.557451 11906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:01 dist-test-slave-x4qh krb5kdc[11558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978061, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:01.560591 11906 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:01:01.561687 11906 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:01.561820 11906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:01.562155 11906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:01.565104 11906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "592ca2c6715649efb4d5295b6edf9505"
format_stamp: "Formatted at 2026-08-06 01:01:01 on dist-test-slave-x4qh"
server_key: "ed7b8f0526b6cf68118b14000985592d"
server_key_iv: "7572e7a7cd0e59748ebe2f6f77ec5834"
server_key_version: "encryptionkey@0"
I20260806 01:01:01.565923 11906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "592ca2c6715649efb4d5295b6edf9505"
format_stamp: "Formatted at 2026-08-06 01:01:01 on dist-test-slave-x4qh"
server_key: "ed7b8f0526b6cf68118b14000985592d"
server_key_iv: "7572e7a7cd0e59748ebe2f6f77ec5834"
server_key_version: "encryptionkey@0"
I20260806 01:01:01.570865 11906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:01:01.574232 11930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:01.575471 11906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:01:01.575671 11906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "592ca2c6715649efb4d5295b6edf9505"
format_stamp: "Formatted at 2026-08-06 01:01:01 on dist-test-slave-x4qh"
server_key: "ed7b8f0526b6cf68118b14000985592d"
server_key_iv: "7572e7a7cd0e59748ebe2f6f77ec5834"
server_key_version: "encryptionkey@0"
I20260806 01:01:01.575896 11906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:01.589917 11906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:01.593998 11906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:01.594285 11906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:01.595162 11906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:01.596423 11906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:01.596540 11906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:01.596675 11906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:01.596762 11906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:01.629136 11906 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:45967
I20260806 01:01:01.629213 12100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:45967 every 8 connection(s)
I20260806 01:01:01.629961 11906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:01.638764 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 11906
I20260806 01:01:01.638988 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:01:01.639582 5554 external_mini_cluster.cc:1428] Setting key c751a52f0c9ce5423ba13e2a23af7307
Aug 06 01:01:01 dist-test-slave-x4qh krb5kdc[11558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978061, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:01.650130 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:01.632130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:37943 (local address 127.5.108.190:36445)
0806 01:01:01.632301 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:01.632306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:01.633346 (+ 1040us) server_negotiation.cc:407] Connection header received
0806 01:01:01.635070 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:01.635074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:01.635137 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:01.635218 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:01.636512 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:01.637182 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:01.638611 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:01.639280 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:01.639428 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:01.643340 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:01.643350 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:01.643358 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:01.643435 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:01.646940 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:01.647724 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:01.647732 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:01.647739 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:01.647899 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:01.648303 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:01.648310 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:01.648315 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:01.648677 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:01.648831 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:01.649428 (+ 597us) server_negotiation.cc:299] Negotiation successful
0806 01:01:01.649681 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:01:01.650681 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:01.632561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36445 (local address 127.5.108.130:37943)
0806 01:01:01.633196 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:01.633229 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:01.634843 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:01.635423 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:01.635431 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:01.635888 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:01.636366 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:01.636380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:01.637309 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:01.637313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:01.638465 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:01.638477 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:01.639526 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:01.639531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:01.639755 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:01.640432 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:01.640481 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:01.643092 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:01.647079 (+ 3987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:01.647085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:01.647095 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:01.647610 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:01.648009 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:01.648012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:01.648014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:01.648183 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:01.648797 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:01.648802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:01.649067 (+ 265us) client_negotiation.cc:769] Sending connection context
0806 01:01:01.649289 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 01:01:01.649623 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":265,"threads_started":1}
I20260806 01:01:01.652256 12102 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36445
I20260806 01:01:01.652546 12102 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:01.653177 12102 heartbeater.cc:511] Master 127.5.108.190:36445 requested a full tablet report, sending...
I20260806 01:01:01.654686 11626 ts_manager.cc:194] Registered new tserver with Master: 592ca2c6715649efb4d5295b6edf9505 (127.5.108.130:45967)
I20260806 01:01:01.655944 11626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:37943
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:01:01.690718 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36445
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:02.013826 12108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:02.014097 12108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:02.014194 12108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:02.020982 12108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:02.021088 12108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:02.021193 12108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:01:02.030102 12108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:02.031210 12108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:02.032548 12108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:02.044816 12116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:02.044785 12119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:02.044862 12117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:02.045843 12108 server_base.cc:1034] running on GCE node
I20260806 01:01:02.046361 12108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:02.047219 12108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:02.048417 12108 hybrid_clock.cc:648] HybridClock initialized: now 1785978062048381 us; error 68 us; skew 500 ppm
I20260806 01:01:02.048707 12108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:02 dist-test-slave-x4qh krb5kdc[11558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978062, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:02.051720 12108 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:01:02.052793 12108 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:02.052937 12108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:02.053252 12108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:02.056046 12108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "86babe2702d445b08ecb99340692fd2a"
format_stamp: "Formatted at 2026-08-06 01:01:02 on dist-test-slave-x4qh"
server_key: "a9f7bd21daf66286b89d5c68434bace8"
server_key_iv: "98777817b692942e730f2b7e3287a706"
server_key_version: "encryptionkey@0"
I20260806 01:01:02.056809 12108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "86babe2702d445b08ecb99340692fd2a"
format_stamp: "Formatted at 2026-08-06 01:01:02 on dist-test-slave-x4qh"
server_key: "a9f7bd21daf66286b89d5c68434bace8"
server_key_iv: "98777817b692942e730f2b7e3287a706"
server_key_version: "encryptionkey@0"
I20260806 01:01:02.061554 12108 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:01:02.065076 12127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:02.066246 12108 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 01:01:02.066412 12108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "86babe2702d445b08ecb99340692fd2a"
format_stamp: "Formatted at 2026-08-06 01:01:02 on dist-test-slave-x4qh"
server_key: "a9f7bd21daf66286b89d5c68434bace8"
server_key_iv: "98777817b692942e730f2b7e3287a706"
server_key_version: "encryptionkey@0"
I20260806 01:01:02.066663 12108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:02.094611 12108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:02.098320 12108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:02.098606 12108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:02.099557 12108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:02.100795 12108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:02.100920 12108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:02.101042 12108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:02.101133 12108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:02.135052 12108 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:43849
I20260806 01:01:02.135142 12283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:43849 every 8 connection(s)
I20260806 01:01:02.136014 12108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:02.139793 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 12108
I20260806 01:01:02.140012 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:01:02.140576 5554 external_mini_cluster.cc:1428] Setting key 83dd970bf0dc48ac92b77642696186c2
Aug 06 01:01:02 dist-test-slave-x4qh krb5kdc[11558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978062, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:02.156847 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.138433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:53839 (local address 127.5.108.190:36445)
0806 01:01:02.138610 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.138614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.139965 (+ 1351us) server_negotiation.cc:407] Connection header received
0806 01:01:02.142257 (+ 2292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.142263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.142353 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.142464 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:02.144496 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.145357 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.147528 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.148027 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.148143 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.151426 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:02.151434 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:02.151439 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:02.151477 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:02.153941 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.154646 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.154650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.154653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.154803 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.155224 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.155228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.155230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.155541 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:02.155711 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.156296 (+ 585us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.156488 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:01:02.157616 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.138811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36445 (local address 127.5.108.131:53839)
0806 01:01:02.139732 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.139792 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:02.141908 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:02.142708 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:02.142719 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:02.143510 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:02.144294 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.144318 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.145503 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.145508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.147403 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.147417 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.148224 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.148229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.148415 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.149076 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:02.149098 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:02.151170 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:02.154079 (+ 2909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:02.154088 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:02.154100 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:02.154506 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:02.154916 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:02.154920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:02.154924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:02.155093 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:02.155683 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:02.155689 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:02.155964 (+ 275us) client_negotiation.cc:769] Sending connection context
0806 01:01:02.156194 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 01:01:02.156647 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":277,"threads_started":1}
I20260806 01:01:02.158779 12285 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36445
I20260806 01:01:02.159005 12285 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:02.159634 12285 heartbeater.cc:511] Master 127.5.108.190:36445 requested a full tablet report, sending...
I20260806 01:01:02.160842 11626 ts_manager.cc:194] Registered new tserver with Master: 86babe2702d445b08ecb99340692fd2a (127.5.108.131:43849)
I20260806 01:01:02.161684 11626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:53839
I20260806 01:01:02.167618 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:01:02.176684 11900 heartbeater.cc:503] Master 127.5.108.190:36445 was elected leader, sending a full tablet report...
I20260806 01:01:02.184435 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.172210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.5.108.190:36445)
0806 01:01:02.172380 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.172384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.172917 (+ 533us) server_negotiation.cc:407] Connection header received
0806 01:01:02.173101 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.173107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.173169 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.173278 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:02.174640 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.175284 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.176446 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.176893 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.177010 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.179223 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:02.179228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:02.179231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:02.179263 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:02.181866 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.182458 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.182465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.182468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.182596 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.182954 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.182958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.182962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.183243 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:02.183519 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.183912 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.184110 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:01:02.187958 11626 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:01:02.194408 11626 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:01:02.215971 12305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.205216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.0.0.1:48014)
0806 01:01:02.206632 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.206688 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:02.206863 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:02.209695 (+ 2832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:02.209699 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:02.209722 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:02.210170 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.210185 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.211993 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.211999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.213725 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.213740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.215082 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.215090 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.215277 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.215310 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:01:02.215441 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 01:01:02.215560 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1182,"thread_start_us":1608,"threads_started":1}
I20260806 01:01:02.215971 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.205833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48014 (local address 127.5.108.129:40649)
0806 01:01:02.209136 (+ 3303us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.209146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.209175 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:01:02.209268 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.209274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.209484 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.210362 (+ 878us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:02.210628 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.211864 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.213866 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.214853 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.215003 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.215224 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.215488 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.215640 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3131,"thread_start_us":2961,"threads_started":1}
I20260806 01:01:02.216007 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.205271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:45967 (local address 127.0.0.1:33506)
0806 01:01:02.206669 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.206687 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:02.206897 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:02.207980 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:02.207990 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:02.208029 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:02.208464 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.208478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.210497 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.210509 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.212894 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.212908 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.214772 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.214779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.215086 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.215177 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 01:01:02.215316 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 01:01:02.215537 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1273,"mutex_wait_us":79,"thread_start_us":991,"threads_started":1}
I20260806 01:01:02.216941 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.205296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33506 (local address 127.5.108.130:45967)
0806 01:01:02.206222 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.206233 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.206792 (+ 559us) server_negotiation.cc:407] Connection header received
0806 01:01:02.207120 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.207128 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.207429 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.207597 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:02.208750 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.210363 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.213088 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.214555 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.214702 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.214876 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.216133 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.216328 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":574,"threads_started":1}
I20260806 01:01:02.217753 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.207193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:43849 (local address 127.0.0.1:33718)
0806 01:01:02.207755 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.207776 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:02.207905 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:02.210375 (+ 2470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:02.210379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:02.210402 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:02.210909 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.210922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.212915 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.212920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.215155 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.215167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.216961 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.216968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.217142 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.217184 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 01:01:02.217268 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 01:01:02.217380 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":5,"thread_start_us":292,"threads_started":1}
I20260806 01:01:02.218701 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.207572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.5.108.131:43849)
0806 01:01:02.209484 (+ 1912us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.209492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.209522 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:01:02.209617 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.209625 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.209894 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.210084 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:02.211165 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.212737 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.215330 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.216771 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.217912 (+ 1141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.218138 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.218214 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.218365 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1761,"thread_start_us":1602,"threads_started":1}
I20260806 01:01:02.219941 11810 tablet_service.cc:1467] Processing CreateTablet for tablet 258730e5f5ca4151a3cd37ade554c096 (DEFAULT_TABLE table=test-table [id=29499a09b1344d1789cc063b8c7d69b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:02.220340 11997 tablet_service.cc:1467] Processing CreateTablet for tablet 258730e5f5ca4151a3cd37ade554c096 (DEFAULT_TABLE table=test-table [id=29499a09b1344d1789cc063b8c7d69b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:02.221311 11810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258730e5f5ca4151a3cd37ade554c096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:02.221786 11997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258730e5f5ca4151a3cd37ade554c096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:02.222148 12198 tablet_service.cc:1467] Processing CreateTablet for tablet 258730e5f5ca4151a3cd37ade554c096 (DEFAULT_TABLE table=test-table [id=29499a09b1344d1789cc063b8c7d69b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:02.223515 12198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258730e5f5ca4151a3cd37ade554c096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:02.232240 12310 tablet_bootstrap.cc:492] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4: Bootstrap starting.
I20260806 01:01:02.235941 12310 tablet_bootstrap.cc:654] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:02.237025 12310 log.cc:824] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:02.237443 12314 tablet_bootstrap.cc:492] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: Bootstrap starting.
I20260806 01:01:02.241441 12313 tablet_bootstrap.cc:492] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: Bootstrap starting.
I20260806 01:01:02.241842 12314 tablet_bootstrap.cc:654] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:02.243119 12314 log.cc:824] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:02.245182 12313 tablet_bootstrap.cc:654] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:02.245350 12310 tablet_bootstrap.cc:492] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4: No bootstrap required, opened a new log
I20260806 01:01:02.245633 12310 ts_tablet_manager.cc:1397] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260806 01:01:02.246215 12313 log.cc:824] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:02.246459 12314 tablet_bootstrap.cc:492] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: No bootstrap required, opened a new log
I20260806 01:01:02.246708 12314 ts_tablet_manager.cc:1397] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 01:01:02.248487 12313 tablet_bootstrap.cc:492] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: No bootstrap required, opened a new log
I20260806 01:01:02.248734 12313 ts_tablet_manager.cc:1397] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 01:01:02.255748 12313 raft_consensus.cc:348] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } }
I20260806 01:01:02.256078 12313 raft_consensus.cc:374] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:02.256204 12313 raft_consensus.cc:729] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 592ca2c6715649efb4d5295b6edf9505, State: Initialized, Role: FOLLOWER
I20260806 01:01:02.256423 12310 raft_consensus.cc:348] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } }
I20260806 01:01:02.256801 12310 raft_consensus.cc:374] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:02.256906 12310 raft_consensus.cc:729] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a739e9bed0483ab01977b661d8fbd4, State: Initialized, Role: FOLLOWER
I20260806 01:01:02.256764 12313 consensus_queue.cc:260] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } }
I20260806 01:01:02.256971 12314 raft_consensus.cc:348] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } }
I20260806 01:01:02.257357 12314 raft_consensus.cc:374] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:02.257459 12314 raft_consensus.cc:729] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86babe2702d445b08ecb99340692fd2a, State: Initialized, Role: FOLLOWER
I20260806 01:01:02.257464 12310 consensus_queue.cc:260] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } }
I20260806 01:01:02.258049 12314 consensus_queue.cc:260] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } }
I20260806 01:01:02.259307 12310 ts_tablet_manager.cc:1428] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4: Time spent starting tablet: real 0.014s user 0.004s sys 0.010s
I20260806 01:01:02.260206 12285 heartbeater.cc:503] Master 127.5.108.190:36445 was elected leader, sending a full tablet report...
I20260806 01:01:02.260854 12314 ts_tablet_manager.cc:1428] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260806 01:01:02.263972 12102 heartbeater.cc:503] Master 127.5.108.190:36445 was elected leader, sending a full tablet report...
I20260806 01:01:02.264118 12324 raft_consensus.cc:482] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:02.264386 12324 raft_consensus.cc:504] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } }
I20260806 01:01:02.265934 12313 ts_tablet_manager.cc:1428] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260806 01:01:02.268054 12324 leader_election.cc:288] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 592ca2c6715649efb4d5295b6edf9505 (127.5.108.130:45967), 03a739e9bed0483ab01977b661d8fbd4 (127.5.108.129:40649)
I20260806 01:01:02.274479 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.269018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:50801 (local address 127.5.108.130:45967)
0806 01:01:02.269215 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.269223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.269240 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:01:02.269293 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.269297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.269372 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.269456 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:02.270285 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.271206 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.272808 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.273590 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.273749 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.273819 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.274105 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.274236 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:01:02.274492 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.268252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:45967 (local address 127.5.108.131:50801)
0806 01:01:02.268592 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.268645 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:02.268780 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:02.269536 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:02.269541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:02.269567 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:02.269806 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.269819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.271327 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.271333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.272639 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.272651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.273743 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.273749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.273943 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.273967 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:01:02.274069 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 01:01:02.274202 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260806 01:01:02.274847 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.268753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:57685 (local address 127.5.108.129:40649)
0806 01:01:02.269012 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.269021 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.269797 (+ 776us) server_negotiation.cc:407] Connection header received
0806 01:01:02.269969 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.269973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.270046 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.270120 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:02.270712 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.271512 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.273152 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.273955 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.274137 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.274207 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.274478 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.274601 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:01:02.274933 12327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.268646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.5.108.131:57685)
0806 01:01:02.269682 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.269698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:02.269826 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:02.270133 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:02.270135 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:02.270165 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:02.270561 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.270570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.271654 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.271659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.272998 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:02.273009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.274158 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.274164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:02.274320 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.274343 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 01:01:02.274452 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 01:01:02.274574 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":553,"threads_started":1}
I20260806 01:01:02.275504 12029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258730e5f5ca4151a3cd37ade554c096" candidate_uuid: "86babe2702d445b08ecb99340692fd2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592ca2c6715649efb4d5295b6edf9505" is_pre_election: true
I20260806 01:01:02.275908 12029 raft_consensus.cc:2454] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86babe2702d445b08ecb99340692fd2a in term 0.
I20260806 01:01:02.275816 11836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258730e5f5ca4151a3cd37ade554c096" candidate_uuid: "86babe2702d445b08ecb99340692fd2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a739e9bed0483ab01977b661d8fbd4" is_pre_election: true
I20260806 01:01:02.276259 11836 raft_consensus.cc:2454] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86babe2702d445b08ecb99340692fd2a in term 0.
I20260806 01:01:02.276520 12128 leader_election.cc:302] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 592ca2c6715649efb4d5295b6edf9505, 86babe2702d445b08ecb99340692fd2a; no voters:
I20260806 01:01:02.276893 12324 raft_consensus.cc:2781] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:01:02.277045 12324 raft_consensus.cc:482] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:01:02.277166 12324 raft_consensus.cc:3037] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:02.279167 12324 raft_consensus.cc:504] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } }
I20260806 01:01:02.279865 12324 leader_election.cc:288] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [CANDIDATE]: Term 1 election: Requested vote from peers 592ca2c6715649efb4d5295b6edf9505 (127.5.108.130:45967), 03a739e9bed0483ab01977b661d8fbd4 (127.5.108.129:40649)
I20260806 01:01:02.280198 12029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258730e5f5ca4151a3cd37ade554c096" candidate_uuid: "86babe2702d445b08ecb99340692fd2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592ca2c6715649efb4d5295b6edf9505"
I20260806 01:01:02.280434 12029 raft_consensus.cc:3037] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:02.280426 11836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258730e5f5ca4151a3cd37ade554c096" candidate_uuid: "86babe2702d445b08ecb99340692fd2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a739e9bed0483ab01977b661d8fbd4"
I20260806 01:01:02.280660 11836 raft_consensus.cc:3037] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:02.282693 11836 raft_consensus.cc:2454] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86babe2702d445b08ecb99340692fd2a in term 1.
I20260806 01:01:02.282819 12029 raft_consensus.cc:2454] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86babe2702d445b08ecb99340692fd2a in term 1.
I20260806 01:01:02.283115 12134 leader_election.cc:302] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a739e9bed0483ab01977b661d8fbd4, 86babe2702d445b08ecb99340692fd2a; no voters:
I20260806 01:01:02.283442 12324 raft_consensus.cc:2781] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:02.283922 12324 raft_consensus.cc:686] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 1 LEADER]: Becoming Leader. State: Replica: 86babe2702d445b08ecb99340692fd2a, State: Running, Role: LEADER
I20260806 01:01:02.284413 12324 consensus_queue.cc:237] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } }
I20260806 01:01:02.288125 11626 catalog_manager.cc:5495] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 86babe2702d445b08ecb99340692fd2a (127.5.108.131). New cstate: current_term: 1 leader_uuid: "86babe2702d445b08ecb99340692fd2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86babe2702d445b08ecb99340692fd2a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43849 } health_report { overall_health: HEALTHY } } }
I20260806 01:01:02.313848 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.309310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.5.108.131:43849)
0806 01:01:02.309486 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.309490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.309505 (+ 15us) server_negotiation.cc:407] Connection header received
0806 01:01:02.309587 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.309590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.309708 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.309851 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:02.310195 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.310868 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.312095 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.312444 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.312530 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.312945 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:02.313394 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:02.313493 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.313564 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.313622 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:01:02.321843 11836 raft_consensus.cc:1261] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [term 1 FOLLOWER]: Refusing update from remote peer 86babe2702d445b08ecb99340692fd2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:02.321875 12029 raft_consensus.cc:1261] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [term 1 FOLLOWER]: Refusing update from remote peer 86babe2702d445b08ecb99340692fd2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:02.322798 12324 consensus_queue.cc:1035] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a739e9bed0483ab01977b661d8fbd4" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 40649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:02.323079 12330 consensus_queue.cc:1035] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "592ca2c6715649efb4d5295b6edf9505" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:02.335244 12336 mvcc.cc:205] Tried to move back new op lower bound from 7315366143261360128 to 7315366143121227776. Current Snapshot: MvccSnapshot[applied={T|T < 7315366143261360128}]
I20260806 01:01:02.348980 11625 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:02.349269 11625 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:02.352730 11625 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82f09a20e85c45fa94df6d8d591f5595: Sending DeleteTablet for 3 replicas of tablet 258730e5f5ca4151a3cd37ade554c096
I20260806 01:01:02.353957 11997 tablet_service.cc:1514] Processing DeleteTablet for tablet 258730e5f5ca4151a3cd37ade554c096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:02 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:33506
I20260806 01:01:02.354259 11810 tablet_service.cc:1514] Processing DeleteTablet for tablet 258730e5f5ca4151a3cd37ade554c096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:02 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:48014
I20260806 01:01:02.355201 12198 tablet_service.cc:1514] Processing DeleteTablet for tablet 258730e5f5ca4151a3cd37ade554c096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:02 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:33718
I20260806 01:01:02.355319 12352 tablet_replica.cc:330] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: stopping tablet replica
I20260806 01:01:02.355299 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 11718
I20260806 01:01:02.355748 12352 raft_consensus.cc:2229] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:02.356150 12359 tablet_replica.cc:330] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: stopping tablet replica
I20260806 01:01:02.356253 12352 raft_consensus.cc:2258] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:02.356662 12359 raft_consensus.cc:2229] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:01:02.357487 12359 raft_consensus.cc:2258] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:02.360064 12359 ts_tablet_manager.cc:1905] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:02.364187 12352 ts_tablet_manager.cc:1905] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:01:02 dist-test-slave-x4qh krb5kdc[11558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:01:02.368706 12359 ts_tablet_manager.cc:1918] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:02.369002 12359 log.cc:1196] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/258730e5f5ca4151a3cd37ade554c096
I20260806 01:01:02.369829 12359 ts_tablet_manager.cc:1939] T 258730e5f5ca4151a3cd37ade554c096 P 86babe2702d445b08ecb99340692fd2a: Deleting consensus metadata
I20260806 01:01:02.371770 11606 catalog_manager.cc:4841] TS 86babe2702d445b08ecb99340692fd2a (127.5.108.131:43849): tablet 258730e5f5ca4151a3cd37ade554c096 (table test-table [id=29499a09b1344d1789cc063b8c7d69b6]) successfully deleted
I20260806 01:01:02.372082 12358 tablet_replica.cc:330] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4: stopping tablet replica
I20260806 01:01:02.372582 12358 raft_consensus.cc:2229] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:02.373139 12358 raft_consensus.cc:2258] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:02.375295 12358 ts_tablet_manager.cc:1905] T 258730e5f5ca4151a3cd37ade554c096 P 03a739e9bed0483ab01977b661d8fbd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:02.375830 12352 ts_tablet_manager.cc:1918] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:02.375996 12352 log.cc:1196] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/258730e5f5ca4151a3cd37ade554c096
I20260806 01:01:02.376556 12352 ts_tablet_manager.cc:1939] T 258730e5f5ca4151a3cd37ade554c096 P 592ca2c6715649efb4d5295b6edf9505: Deleting consensus metadata
I20260806 01:01:02.378293 11600 catalog_manager.cc:4841] TS 592ca2c6715649efb4d5295b6edf9505 (127.5.108.130:45967): tablet 258730e5f5ca4151a3cd37ade554c096 (table test-table [id=29499a09b1344d1789cc063b8c7d69b6]) successfully deleted
I20260806 01:01:02.378945 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.357417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.5.108.129:40649)
0806 01:01:02.357640 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.357650 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.358064 (+ 414us) server_negotiation.cc:407] Connection header received
0806 01:01:02.358337 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.358343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.358421 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.358508 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:02.360245 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.361407 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.363476 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.364161 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.364275 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.369655 (+ 5380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:02.369671 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:02.369688 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:02.369729 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:02.374271 (+ 4542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.375248 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.375256 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.375259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.375531 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.376144 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.376162 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.376165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.376669 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:02.376982 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.378176 (+ 1194us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.378406 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:01:02.379894 11891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:02.485236 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 11718
I20260806 01:01:02.510282 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 11906
W20260806 01:01:02.510282 11606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.108.129:40649 (error 108)
I20260806 01:01:02.512547 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.511979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.0.0.1:48026)
0806 01:01:02.512200 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.512377 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:40649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 01:01:02.512815 11606 catalog_manager.cc:4584] TS 03a739e9bed0483ab01977b661d8fbd4 (127.5.108.129:40649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 258730e5f5ca4151a3cd37ade554c096: Network error: Client connection negotiation failed: client connection to 127.5.108.129:40649: connect: Connection refused (error 111)
Aug 06 01:01:02 dist-test-slave-x4qh krb5kdc[11558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:01:02.526508 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.511605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.5.108.130:45967)
0806 01:01:02.511804 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.511811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.512312 (+ 501us) server_negotiation.cc:407] Connection header received
0806 01:01:02.512523 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.512527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.512593 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.512709 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:02.514155 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.514941 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.516300 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.516690 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.516822 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.520973 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:02.520986 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:02.520999 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:02.521028 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:02.523733 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.524439 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.524446 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.524448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.524574 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.524970 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.524974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.524976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.525324 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:02.525648 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.525918 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.526142 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:01:02.527221 12092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:02.566740 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.566223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.0.0.1:48042)
0806 01:01:02.566449 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.566555 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:40649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 01:01:02.567938 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.567511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.0.0.1:48046)
0806 01:01:02.567683 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.567793 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:40649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 01:01:02.626377 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 11906
I20260806 01:01:02.636597 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.636033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.0.0.1:48054)
0806 01:01:02.636290 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.636397 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:40649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 01:01:02.637794 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.637351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.0.0.1:48070)
0806 01:01:02.637580 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.637661 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:40649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 01:01:02.651434 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 12108
Aug 06 01:01:02 dist-test-slave-x4qh krb5kdc[11558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:01:02.670046 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.652222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33732 (local address 127.5.108.131:43849)
0806 01:01:02.652396 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:02.652401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:02.652418 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:01:02.652594 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:02.652598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:02.652700 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:02.652778 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:02.654691 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.655602 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.657426 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:02.657879 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:02.657970 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:02.663140 (+ 5170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:02.663154 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:02.663173 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:02.663225 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:02.667147 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.667806 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.667815 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.667817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.667997 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:02.668339 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:02.668344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:02.668347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:02.668834 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:02.669171 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:02.669365 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 01:01:02.669592 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:01:02.670938 12279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:02.718290 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.717869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.0.0.1:48086)
0806 01:01:02.718044 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.718140 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:40649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 01:01:02.719287 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.718910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.0.0.1:48102)
0806 01:01:02.719104 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.719178 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:40649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 01:01:02.789232 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 12108
I20260806 01:01:02.816970 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 11578
I20260806 01:01:02.818116 11668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:02.921458 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:02.920904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:40649 (local address 127.0.0.1:48118)
0806 01:01:02.921157 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:02.921257 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:40649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 01:01:02.921443 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 11578
2026-08-06T01:01:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4231 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T01:01:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:02Z Disabled control of system clock
I20260806 01:01:03.000506 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:42691
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:44417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:42691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 01:01:03.308874 12370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:03.309168 12370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:03.309252 12370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:03.316187 12370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:03.316270 12370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:03.316304 12370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:03.316366 12370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:03.316407 12370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:03.324421 12370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:44417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:42691
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:42691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:03.325512 12370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:03.327250 12370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:03.339066 12376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:03.339170 12377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:03.339280 12381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:03.340400 12370 server_base.cc:1034] running on GCE node
I20260806 01:01:03.341154 12370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:03.342468 12370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:03.343686 12370 hybrid_clock.cc:648] HybridClock initialized: now 1785978063343633 us; error 41 us; skew 500 ppm
I20260806 01:01:03.344039 12370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:03.346302 12370 webserver.cc:460] Webserver started at http://127.5.108.190:36955/ using document root <none> and password file <none>
I20260806 01:01:03.347007 12370 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:03.347144 12370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:03.347539 12370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:03.350275 12370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "cda6e986ad53499ea87e36ed7de8d60d"
format_stamp: "Formatted at 2026-08-06 01:01:03 on dist-test-slave-x4qh"
server_key: "5bfe18b1673cf54885623e54dbfe96f4"
server_key_iv: "fb3d980673f3dfffb2c5ccb152f18596"
server_key_version: "encryptionkey@0"
I20260806 01:01:03.351083 12370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "cda6e986ad53499ea87e36ed7de8d60d"
format_stamp: "Formatted at 2026-08-06 01:01:03 on dist-test-slave-x4qh"
server_key: "5bfe18b1673cf54885623e54dbfe96f4"
server_key_iv: "fb3d980673f3dfffb2c5ccb152f18596"
server_key_version: "encryptionkey@0"
I20260806 01:01:03.356158 12370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 01:01:03.359475 12391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:03.360721 12370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:01:03.360914 12370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "cda6e986ad53499ea87e36ed7de8d60d"
format_stamp: "Formatted at 2026-08-06 01:01:03 on dist-test-slave-x4qh"
server_key: "5bfe18b1673cf54885623e54dbfe96f4"
server_key_iv: "fb3d980673f3dfffb2c5ccb152f18596"
server_key_version: "encryptionkey@0"
I20260806 01:01:03.361124 12370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:03.366981 12370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:03.367841 12370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:03.368144 12370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:03.387015 12370 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:42691
I20260806 01:01:03.387050 12457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:42691 every 8 connection(s)
I20260806 01:01:03.388274 12370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:03.391901 12458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:03.397258 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 12370
I20260806 01:01:03.397471 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:03.398108 5554 external_mini_cluster.cc:1428] Setting key 71d4329b4d16df62af48147ef1d4bcde
I20260806 01:01:03.403357 12458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d: Bootstrap starting.
I20260806 01:01:03.407384 12458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:03.407450 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:03.400145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.5.108.190:42691)
0806 01:01:03.401259 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:03.401272 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:03.401321 (+ 49us) server_negotiation.cc:407] Connection header received
0806 01:01:03.403461 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:03.403490 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:03.404033 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:03.404444 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:03.405532 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:03.405539 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:03.405555 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:03.405574 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:03.405660 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:03.405693 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:03.405910 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:03.406044 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 01:01:03.406284 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":380,"threads_started":1}
I20260806 01:01:03.408456 12458 log.cc:824] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:03.411191 12458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d: No bootstrap required, opened a new log
I20260806 01:01:03.418120 12458 raft_consensus.cc:348] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda6e986ad53499ea87e36ed7de8d60d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42691 } }
I20260806 01:01:03.418413 12458 raft_consensus.cc:374] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:03.418497 12458 raft_consensus.cc:729] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cda6e986ad53499ea87e36ed7de8d60d, State: Initialized, Role: FOLLOWER
I20260806 01:01:03.419003 12458 consensus_queue.cc:260] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda6e986ad53499ea87e36ed7de8d60d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42691 } }
I20260806 01:01:03.419238 12458 raft_consensus.cc:388] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:03.419413 12458 raft_consensus.cc:482] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:03.419587 12458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:03.421622 12458 raft_consensus.cc:504] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda6e986ad53499ea87e36ed7de8d60d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42691 } }
I20260806 01:01:03.422057 12458 leader_election.cc:302] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cda6e986ad53499ea87e36ed7de8d60d; no voters:
I20260806 01:01:03.422617 12458 leader_election.cc:288] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:03.422719 12466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:03.424057 12466 raft_consensus.cc:686] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [term 1 LEADER]: Becoming Leader. State: Replica: cda6e986ad53499ea87e36ed7de8d60d, State: Running, Role: LEADER
I20260806 01:01:03.424571 12466 consensus_queue.cc:237] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda6e986ad53499ea87e36ed7de8d60d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42691 } }
I20260806 01:01:03.424674 12458 sys_catalog.cc:564] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:03.427693 12470 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cda6e986ad53499ea87e36ed7de8d60d. Latest consensus state: current_term: 1 leader_uuid: "cda6e986ad53499ea87e36ed7de8d60d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda6e986ad53499ea87e36ed7de8d60d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42691 } } }
I20260806 01:01:03.427716 12469 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cda6e986ad53499ea87e36ed7de8d60d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda6e986ad53499ea87e36ed7de8d60d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 42691 } } }
I20260806 01:01:03.428210 12469 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:03.428205 12470 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:03.429862 12480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:03.435243 12480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:03.443040 12480 catalog_manager.cc:1347] Generated new cluster ID: 5f92af1f77a449abbb7b2969c5765f01
I20260806 01:01:03.443161 12480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:03.459535 12480 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:03.459954 12480 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:03.466356 12480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d: Generated new TSK 0
I20260806 01:01:03.467181 12480 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:01:03.478227 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42691
--builtin_ntp_servers=127.5.108.148:44417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 01:01:03.814124 12495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:03.814383 12495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:03.814473 12495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:03.821338 12495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:03.821429 12495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:03.821530 12495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:03.830585 12495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:44417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:03.831799 12495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:03.833559 12495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:03.846701 12501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:03.846769 12506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:03.846817 12502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:03.857008 12495 server_base.cc:1034] running on GCE node
I20260806 01:01:03.857617 12495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:03.858553 12495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:03.859797 12495 hybrid_clock.cc:648] HybridClock initialized: now 1785978063859765 us; error 53 us; skew 500 ppm
I20260806 01:01:03.860097 12495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:03.862313 12495 webserver.cc:460] Webserver started at http://127.5.108.129:36237/ using document root <none> and password file <none>
I20260806 01:01:03.862989 12495 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:03.863121 12495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:03.863507 12495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:03.866674 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "921fe885be854231b41f0c0b3d853611"
format_stamp: "Formatted at 2026-08-06 01:01:03 on dist-test-slave-x4qh"
server_key: "f3359ee1ef7af9ba1a2a95f05f2e8c48"
server_key_iv: "292381b0221b8ba5bb1332c5280f2da2"
server_key_version: "encryptionkey@0"
I20260806 01:01:03.867506 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "921fe885be854231b41f0c0b3d853611"
format_stamp: "Formatted at 2026-08-06 01:01:03 on dist-test-slave-x4qh"
server_key: "f3359ee1ef7af9ba1a2a95f05f2e8c48"
server_key_iv: "292381b0221b8ba5bb1332c5280f2da2"
server_key_version: "encryptionkey@0"
I20260806 01:01:03.872434 12495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:01:03.875722 12513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:03.876961 12495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:01:03.877149 12495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "921fe885be854231b41f0c0b3d853611"
format_stamp: "Formatted at 2026-08-06 01:01:03 on dist-test-slave-x4qh"
server_key: "f3359ee1ef7af9ba1a2a95f05f2e8c48"
server_key_iv: "292381b0221b8ba5bb1332c5280f2da2"
server_key_version: "encryptionkey@0"
I20260806 01:01:03.877377 12495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:03.882918 12495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:03.883729 12495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:03.884028 12495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:03.885222 12495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:03.886636 12495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:03.886780 12495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:03.886909 12495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:03.887001 12495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:03.920012 12495 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:44559
I20260806 01:01:03.920096 12679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:44559 every 8 connection(s)
I20260806 01:01:03.921267 12495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:03.929639 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:03.923747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:39741 (local address 127.5.108.190:42691)
0806 01:01:03.924242 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:03.924247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:03.925403 (+ 1156us) server_negotiation.cc:407] Connection header received
0806 01:01:03.927228 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:03.927233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:03.927309 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:03.927465 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:03.928731 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:03.928738 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:03.928742 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:03.928761 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:03.928816 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:03.928828 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:03.928975 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:03.929195 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 01:01:03.929286 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":280,"threads_started":1}
I20260806 01:01:03.930382 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:03.924242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:42691 (local address 127.5.108.129:39741)
0806 01:01:03.925247 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:03.925289 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:03.926964 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:03.927700 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:03.927708 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:03.927801 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:03.928436 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:03.928438 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:03.928473 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:03.928550 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:03.928941 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:03.928954 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:03.928962 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 01:01:03.929245 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 01:01:03.929517 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":340,"threads_started":1}
I20260806 01:01:03.931105 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 12495
I20260806 01:01:03.931330 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:01:03.931967 12680 heartbeater.cc:347] Connected to a master server at 127.5.108.190:42691
I20260806 01:01:03.932142 5554 external_mini_cluster.cc:1428] Setting key d91fb4cbc550d3903000bfda7504a662
I20260806 01:01:03.932260 12680 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:03.932817 12680 heartbeater.cc:511] Master 127.5.108.190:42691 requested a full tablet report, sending...
I20260806 01:01:03.934904 12411 ts_manager.cc:194] Registered new tserver with Master: 921fe885be854231b41f0c0b3d853611 (127.5.108.129:44559)
I20260806 01:01:03.934971 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42691
--builtin_ntp_servers=127.5.108.148:44417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 01:01:04.249140 12686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:04.249377 12686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:04.249526 12686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:04.256613 12686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:04.256703 12686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:04.256896 12686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:04.265734 12686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:44417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:04.266803 12686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:04.268518 12686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:04.280958 12691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:04.281008 12697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:04.281075 12694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:04.284772 12686 server_base.cc:1034] running on GCE node
I20260806 01:01:04.285287 12686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:04.286147 12686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:04.287345 12686 hybrid_clock.cc:648] HybridClock initialized: now 1785978064287305 us; error 36 us; skew 500 ppm
I20260806 01:01:04.287711 12686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:04.289947 12686 webserver.cc:460] Webserver started at http://127.5.108.130:41439/ using document root <none> and password file <none>
I20260806 01:01:04.290623 12686 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:04.290748 12686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:04.291076 12686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:04.293883 12686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "f03a4e9a009d404a98627b333ac0aae3"
format_stamp: "Formatted at 2026-08-06 01:01:04 on dist-test-slave-x4qh"
server_key: "2e61e9d362975edec3d1f8f3f614a8b4"
server_key_iv: "ffc26f3ac97b5930c10269c86f991616"
server_key_version: "encryptionkey@0"
I20260806 01:01:04.294680 12686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "f03a4e9a009d404a98627b333ac0aae3"
format_stamp: "Formatted at 2026-08-06 01:01:04 on dist-test-slave-x4qh"
server_key: "2e61e9d362975edec3d1f8f3f614a8b4"
server_key_iv: "ffc26f3ac97b5930c10269c86f991616"
server_key_version: "encryptionkey@0"
I20260806 01:01:04.299607 12686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 01:01:04.302708 12707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:04.304033 12686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:04.304212 12686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "f03a4e9a009d404a98627b333ac0aae3"
format_stamp: "Formatted at 2026-08-06 01:01:04 on dist-test-slave-x4qh"
server_key: "2e61e9d362975edec3d1f8f3f614a8b4"
server_key_iv: "ffc26f3ac97b5930c10269c86f991616"
server_key_version: "encryptionkey@0"
I20260806 01:01:04.304466 12686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:04.310156 12686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:04.310879 12686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:04.311141 12686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:04.312069 12686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:04.313297 12686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:04.313423 12686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:04.313549 12686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:04.313639 12686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:04.346283 12686 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:44979
I20260806 01:01:04.346356 12863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:44979 every 8 connection(s)
I20260806 01:01:04.347476 12686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:04.355031 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.349828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:42883 (local address 127.5.108.190:42691)
0806 01:01:04.349989 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:04.349993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:04.351214 (+ 1221us) server_negotiation.cc:407] Connection header received
0806 01:01:04.352802 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:04.352807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:04.352907 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:04.353017 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:04.354239 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:04.354245 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:04.354250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:04.354272 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:04.354310 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:04.354321 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:04.354462 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:04.354659 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 01:01:04.354730 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:01:04.355858 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.350255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:42691 (local address 127.5.108.130:42883)
0806 01:01:04.351067 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:04.351098 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:04.352546 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:04.353238 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:04.353244 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:04.353320 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:04.353945 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:04.353947 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:04.353988 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:04.354064 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:04.354421 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:04.354433 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:04.354443 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:01:04.354725 (+ 282us) client_negotiation.cc:240] Negotiation successful
0806 01:01:04.354943 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":248,"threads_started":1}
I20260806 01:01:04.357115 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 12686
I20260806 01:01:04.357156 12864 heartbeater.cc:347] Connected to a master server at 127.5.108.190:42691
I20260806 01:01:04.357316 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:01:04.357471 12864 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:04.357807 5554 external_mini_cluster.cc:1428] Setting key 044bc3f948bd74f4e9fbd2d9dc3e829e
I20260806 01:01:04.358003 12864 heartbeater.cc:511] Master 127.5.108.190:42691 requested a full tablet report, sending...
I20260806 01:01:04.359412 12410 ts_manager.cc:194] Registered new tserver with Master: f03a4e9a009d404a98627b333ac0aae3 (127.5.108.130:44979)
I20260806 01:01:04.360729 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42691
--builtin_ntp_servers=127.5.108.148:44417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 01:01:04.694533 12869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:04.694782 12869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:04.694939 12869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:04.704288 12869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:04.704473 12869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:04.704677 12869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:01:04.714602 12869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:44417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:42691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:04.715842 12869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:04.717530 12869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:04.730640 12875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:04.730576 12879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:04.730675 12877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:04.731765 12869 server_base.cc:1034] running on GCE node
I20260806 01:01:04.732347 12869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:04.733156 12869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:04.734359 12869 hybrid_clock.cc:648] HybridClock initialized: now 1785978064734308 us; error 47 us; skew 500 ppm
I20260806 01:01:04.734664 12869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:04.736948 12869 webserver.cc:460] Webserver started at http://127.5.108.131:33103/ using document root <none> and password file <none>
I20260806 01:01:04.737635 12869 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:04.737753 12869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:04.738068 12869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:04.740813 12869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "a58a7ac42b344525847da93c640a1eae"
format_stamp: "Formatted at 2026-08-06 01:01:04 on dist-test-slave-x4qh"
server_key: "69e8cfbe49981adf699ffa5198c794b7"
server_key_iv: "6a83e1f6fa7a282789fc90c2c3e1e9ef"
server_key_version: "encryptionkey@0"
I20260806 01:01:04.741611 12869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "a58a7ac42b344525847da93c640a1eae"
format_stamp: "Formatted at 2026-08-06 01:01:04 on dist-test-slave-x4qh"
server_key: "69e8cfbe49981adf699ffa5198c794b7"
server_key_iv: "6a83e1f6fa7a282789fc90c2c3e1e9ef"
server_key_version: "encryptionkey@0"
I20260806 01:01:04.746232 12869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 01:01:04.749320 12889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:04.750427 12869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 01:01:04.750597 12869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "a58a7ac42b344525847da93c640a1eae"
format_stamp: "Formatted at 2026-08-06 01:01:04 on dist-test-slave-x4qh"
server_key: "69e8cfbe49981adf699ffa5198c794b7"
server_key_iv: "6a83e1f6fa7a282789fc90c2c3e1e9ef"
server_key_version: "encryptionkey@0"
I20260806 01:01:04.750814 12869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:04.756083 12869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:04.756754 12869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:04.756994 12869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:04.757804 12869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:04.759131 12869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:04.759265 12869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:04.759402 12869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:04.759482 12869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 01:01:04.792498 12869 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:34521
I20260806 01:01:04.792578 13043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:34521 every 8 connection(s)
I20260806 01:01:04.793717 12869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:04.801505 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.796255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:59519 (local address 127.5.108.190:42691)
0806 01:01:04.796442 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:04.796446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:04.797683 (+ 1237us) server_negotiation.cc:407] Connection header received
0806 01:01:04.799332 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:04.799337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:04.799430 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:04.799536 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:04.800796 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:04.800801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:04.800806 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:04.800824 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:04.800865 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:04.800876 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:04.800991 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:04.801182 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 01:01:04.801248 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:01:04.802266 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.796684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:42691 (local address 127.5.108.131:59519)
0806 01:01:04.797541 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:04.797573 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:04.799031 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:04.799740 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:04.799747 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:04.799840 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:04.800526 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:04.800528 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:04.800561 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:04.800636 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:04.800967 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:04.800978 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:04.800984 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 01:01:04.801242 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 01:01:04.801443 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":275,"threads_started":1}
I20260806 01:01:04.803529 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 12869
I20260806 01:01:04.803627 13044 heartbeater.cc:347] Connected to a master server at 127.5.108.190:42691
I20260806 01:01:04.803691 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:01:04.803929 13044 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:04.804209 5554 external_mini_cluster.cc:1428] Setting key 43c2e59463b230f543b5d07bb2edbe9d
I20260806 01:01:04.804524 13044 heartbeater.cc:511] Master 127.5.108.190:42691 requested a full tablet report, sending...
I20260806 01:01:04.806021 12410 ts_manager.cc:194] Registered new tserver with Master: a58a7ac42b344525847da93c640a1eae (127.5.108.131:34521)
I20260806 01:01:04.807278 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:01:04.813840 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.811652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47250 (local address 127.5.108.190:42691)
0806 01:01:04.811854 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:04.811862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:04.812343 (+ 481us) server_negotiation.cc:407] Connection header received
0806 01:01:04.812441 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:04.812444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:04.812486 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:04.812556 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:04.813339 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:04.813342 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:04.813345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:04.813357 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:04.813382 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:04.813391 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:04.813478 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:04.813536 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 01:01:04.813572 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:01:04.817534 12411 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:01:04.824615 12411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:01:04.841194 13061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.836039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44979 (local address 127.0.0.1:47280)
0806 01:01:04.837024 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:04.837066 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:04.837202 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:04.838601 (+ 1399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:04.838609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:04.838689 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:04.839406 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:04.839409 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:04.839438 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:04.839518 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:04.840548 (+ 1030us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:04.840569 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:04.840576 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:01:04.840747 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 01:01:04.840825 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":586,"threads_started":1}
I20260806 01:01:04.841203 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.836553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.5.108.130:44979)
0806 01:01:04.837405 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:04.837411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:04.837433 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:01:04.837518 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:04.837538 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:04.837754 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:04.837908 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:04.840282 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:04.840292 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:04.840305 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:04.840332 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:04.840390 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:04.840411 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:04.840634 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:04.840735 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 01:01:04.840832 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":545,"threads_started":1}
I20260806 01:01:04.841922 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.837413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:34521 (local address 127.0.0.1:56458)
0806 01:01:04.837983 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:04.837999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:04.838085 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:04.839154 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:04.839158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:04.839179 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:04.839887 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:04.839893 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:04.839931 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:04.839990 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:04.841233 (+ 1243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:04.841237 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:04.841242 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 01:01:04.841347 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 01:01:04.841389 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":268,"threads_started":1}
I20260806 01:01:04.842773 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.837684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.5.108.131:34521)
0806 01:01:04.838618 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:04.838632 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:04.838663 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:01:04.838732 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:04.838755 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:04.838958 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:04.839639 (+ 681us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:04.840995 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:04.841004 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:04.841016 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:04.841032 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:04.841075 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:04.841095 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:04.842222 (+ 1127us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:04.842291 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 01:01:04.842411 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":792,"thread_start_us":623,"threads_started":1}
I20260806 01:01:04.844043 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.838024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44559 (local address 127.0.0.1:44682)
0806 01:01:04.840278 (+ 2254us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:04.840295 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:04.840378 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:04.841662 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:04.841665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:04.841681 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:04.842805 (+ 1124us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:04.842809 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:04.842830 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:04.842877 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:04.843400 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:04.843404 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:04.843409 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 01:01:04.843551 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 01:01:04.843603 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2127,"thread_start_us":1974,"threads_started":1}
I20260806 01:01:04.844723 13071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.838551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.5.108.129:44559)
0806 01:01:04.839853 (+ 1302us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:04.839862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:04.840483 (+ 621us) server_negotiation.cc:407] Connection header received
0806 01:01:04.840565 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:04.840571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:04.840745 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:04.840867 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:04.843051 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:04.843058 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:04.843069 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:04.843088 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:04.843146 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:04.843166 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:04.843315 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:04.844316 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0806 01:01:04.844429 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1131,"thread_start_us":580,"threads_started":1}
I20260806 01:01:04.845577 12778 tablet_service.cc:1467] Processing CreateTablet for tablet a8cb6b4b86fb44278ecf895c461bceaa (DEFAULT_TABLE table=test-table [id=11d19b05de0a49d0b3a9b3d8e927fd3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:04.846009 12952 tablet_service.cc:1467] Processing CreateTablet for tablet a8cb6b4b86fb44278ecf895c461bceaa (DEFAULT_TABLE table=test-table [id=11d19b05de0a49d0b3a9b3d8e927fd3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:04.847067 12778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8cb6b4b86fb44278ecf895c461bceaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:04.847250 12952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8cb6b4b86fb44278ecf895c461bceaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:04.848313 12587 tablet_service.cc:1467] Processing CreateTablet for tablet a8cb6b4b86fb44278ecf895c461bceaa (DEFAULT_TABLE table=test-table [id=11d19b05de0a49d0b3a9b3d8e927fd3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:04.849627 12587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8cb6b4b86fb44278ecf895c461bceaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:04.859910 13073 tablet_bootstrap.cc:492] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: Bootstrap starting.
I20260806 01:01:04.861624 13076 tablet_bootstrap.cc:492] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: Bootstrap starting.
I20260806 01:01:04.863340 13077 tablet_bootstrap.cc:492] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611: Bootstrap starting.
I20260806 01:01:04.863359 13073 tablet_bootstrap.cc:654] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:04.864629 13073 log.cc:824] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:04.864805 13076 tablet_bootstrap.cc:654] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:04.866057 13076 log.cc:824] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:04.866542 13077 tablet_bootstrap.cc:654] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:04.867691 13077 log.cc:824] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:04.868208 13073 tablet_bootstrap.cc:492] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: No bootstrap required, opened a new log
I20260806 01:01:04.868489 13073 ts_tablet_manager.cc:1397] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260806 01:01:04.868517 13076 tablet_bootstrap.cc:492] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: No bootstrap required, opened a new log
I20260806 01:01:04.868748 13076 ts_tablet_manager.cc:1397] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 01:01:04.870374 13077 tablet_bootstrap.cc:492] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611: No bootstrap required, opened a new log
I20260806 01:01:04.870597 13077 ts_tablet_manager.cc:1397] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260806 01:01:04.875643 13076 raft_consensus.cc:348] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } }
I20260806 01:01:04.876119 13076 raft_consensus.cc:374] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:04.876269 13076 raft_consensus.cc:729] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03a4e9a009d404a98627b333ac0aae3, State: Initialized, Role: FOLLOWER
I20260806 01:01:04.876843 13076 consensus_queue.cc:260] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } }
I20260806 01:01:04.877892 12864 heartbeater.cc:503] Master 127.5.108.190:42691 was elected leader, sending a full tablet report...
I20260806 01:01:04.878332 13076 ts_tablet_manager.cc:1428] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: Time spent starting tablet: real 0.009s user 0.006s sys 0.004s
I20260806 01:01:04.878278 13073 raft_consensus.cc:348] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } }
I20260806 01:01:04.878741 13073 raft_consensus.cc:374] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:04.878854 13073 raft_consensus.cc:729] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58a7ac42b344525847da93c640a1eae, State: Initialized, Role: FOLLOWER
I20260806 01:01:04.879453 13073 consensus_queue.cc:260] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } }
I20260806 01:01:04.880735 13044 heartbeater.cc:503] Master 127.5.108.190:42691 was elected leader, sending a full tablet report...
I20260806 01:01:04.881211 13077 raft_consensus.cc:348] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } }
I20260806 01:01:04.881507 13077 raft_consensus.cc:374] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:04.881621 13077 raft_consensus.cc:729] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921fe885be854231b41f0c0b3d853611, State: Initialized, Role: FOLLOWER
I20260806 01:01:04.882274 13077 consensus_queue.cc:260] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } }
I20260806 01:01:04.882467 13073 ts_tablet_manager.cc:1428] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
I20260806 01:01:04.883723 13077 ts_tablet_manager.cc:1428] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260806 01:01:04.883948 12680 heartbeater.cc:503] Master 127.5.108.190:42691 was elected leader, sending a full tablet report...
W20260806 01:01:04.923590 12682 tablet.cc:2369] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:01:04.925719 13084 raft_consensus.cc:482] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:04.926011 13084 raft_consensus.cc:504] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } }
I20260806 01:01:04.927595 13084 leader_election.cc:288] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f03a4e9a009d404a98627b333ac0aae3 (127.5.108.130:44979), 921fe885be854231b41f0c0b3d853611 (127.5.108.129:44559)
I20260806 01:01:04.930341 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.928032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:46187 (local address 127.5.108.130:44979)
0806 01:01:04.928228 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:04.928236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:04.928255 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:01:04.928317 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:04.928323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:04.928382 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:04.928478 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:04.929662 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:04.929667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:04.929671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:04.929691 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:04.929728 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:04.929746 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:04.929857 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:04.930046 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 01:01:04.930104 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 01:01:04.930358 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.927876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44979 (local address 127.5.108.131:46187)
0806 01:01:04.928057 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:04.928079 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:04.928183 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:04.928654 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:04.928660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:04.928684 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:04.929432 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:04.929437 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:04.929455 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:04.929507 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:04.929912 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:04.929919 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:04.929923 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 01:01:04.930042 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 01:01:04.930096 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 01:01:04.931438 13071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.928537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:35611 (local address 127.5.108.129:44559)
0806 01:01:04.928707 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:04.928714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:04.929335 (+ 621us) server_negotiation.cc:407] Connection header received
0806 01:01:04.929505 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:04.929509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:04.929582 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:04.929680 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:04.930662 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:04.930668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:04.930673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:04.930691 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:04.930725 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:04.930739 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:04.930815 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:04.931049 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 01:01:04.931115 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:01:04.931469 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.928241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44559 (local address 127.5.108.131:35611)
0806 01:01:04.929222 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:04.929247 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:04.929376 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:04.929702 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:04.929706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:04.929727 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:04.930431 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:04.930434 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:04.930450 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:04.930499 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:04.930890 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:04.930895 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:04.930900 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 01:01:04.931019 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 01:01:04.931064 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":634,"threads_started":1}
I20260806 01:01:04.931421 12802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8cb6b4b86fb44278ecf895c461bceaa" candidate_uuid: "a58a7ac42b344525847da93c640a1eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03a4e9a009d404a98627b333ac0aae3" is_pre_election: true
I20260806 01:01:04.931978 12802 raft_consensus.cc:2454] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a58a7ac42b344525847da93c640a1eae in term 0.
I20260806 01:01:04.932724 12893 leader_election.cc:302] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a58a7ac42b344525847da93c640a1eae, f03a4e9a009d404a98627b333ac0aae3; no voters:
I20260806 01:01:04.932596 12615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8cb6b4b86fb44278ecf895c461bceaa" candidate_uuid: "a58a7ac42b344525847da93c640a1eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921fe885be854231b41f0c0b3d853611" is_pre_election: true
I20260806 01:01:04.933140 13084 raft_consensus.cc:2781] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:01:04.933137 12615 raft_consensus.cc:2454] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a58a7ac42b344525847da93c640a1eae in term 0.
I20260806 01:01:04.933261 13084 raft_consensus.cc:482] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:01:04.933329 13084 raft_consensus.cc:3037] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:04.936102 13084 raft_consensus.cc:504] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } }
I20260806 01:01:04.936646 13084 leader_election.cc:288] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [CANDIDATE]: Term 1 election: Requested vote from peers f03a4e9a009d404a98627b333ac0aae3 (127.5.108.130:44979), 921fe885be854231b41f0c0b3d853611 (127.5.108.129:44559)
I20260806 01:01:04.936995 12802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8cb6b4b86fb44278ecf895c461bceaa" candidate_uuid: "a58a7ac42b344525847da93c640a1eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03a4e9a009d404a98627b333ac0aae3"
I20260806 01:01:04.937054 12615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8cb6b4b86fb44278ecf895c461bceaa" candidate_uuid: "a58a7ac42b344525847da93c640a1eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921fe885be854231b41f0c0b3d853611"
I20260806 01:01:04.937220 12802 raft_consensus.cc:3037] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:04.937228 12615 raft_consensus.cc:3037] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:04.939108 12615 raft_consensus.cc:2454] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a58a7ac42b344525847da93c640a1eae in term 1.
I20260806 01:01:04.939164 12802 raft_consensus.cc:2454] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a58a7ac42b344525847da93c640a1eae in term 1.
I20260806 01:01:04.939478 12893 leader_election.cc:302] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921fe885be854231b41f0c0b3d853611, a58a7ac42b344525847da93c640a1eae; no voters:
I20260806 01:01:04.939731 13084 raft_consensus.cc:2781] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:04.940192 13084 raft_consensus.cc:686] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 1 LEADER]: Becoming Leader. State: Replica: a58a7ac42b344525847da93c640a1eae, State: Running, Role: LEADER
I20260806 01:01:04.940665 13084 consensus_queue.cc:237] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } }
I20260806 01:01:04.944983 12411 catalog_manager.cc:5495] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae reported cstate change: term changed from 0 to 1, leader changed from <none> to a58a7ac42b344525847da93c640a1eae (127.5.108.131). New cstate: current_term: 1 leader_uuid: "a58a7ac42b344525847da93c640a1eae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a58a7ac42b344525847da93c640a1eae" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 } health_report { overall_health: UNKNOWN } } }
I20260806 01:01:04.964519 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:04.962298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56462 (local address 127.5.108.131:34521)
0806 01:01:04.962476 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:04.962481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:04.962626 (+ 145us) server_negotiation.cc:407] Connection header received
0806 01:01:04.962855 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:04.962858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:04.962932 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:04.963028 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:04.963869 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:04.963874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:04.963878 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:04.963916 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:04.963946 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:04.963957 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:04.964084 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:04.964192 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 01:01:04.964251 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 01:01:04.971767 12615 raft_consensus.cc:1261] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [term 1 FOLLOWER]: Refusing update from remote peer a58a7ac42b344525847da93c640a1eae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:04.971881 12802 raft_consensus.cc:1261] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [term 1 FOLLOWER]: Refusing update from remote peer a58a7ac42b344525847da93c640a1eae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:04.972712 13088 consensus_queue.cc:1035] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "921fe885be854231b41f0c0b3d853611" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:04.973053 13084 consensus_queue.cc:1035] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f03a4e9a009d404a98627b333ac0aae3" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:04.998800 12411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:04.999053 12411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:05.002311 12411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cda6e986ad53499ea87e36ed7de8d60d: Sending DeleteTablet for 3 replicas of tablet a8cb6b4b86fb44278ecf895c461bceaa
I20260806 01:01:05.003381 12778 tablet_service.cc:1514] Processing DeleteTablet for tablet a8cb6b4b86fb44278ecf895c461bceaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:04 UTC) from {username='slave'} at 127.0.0.1:47280
I20260806 01:01:05.003580 12952 tablet_service.cc:1514] Processing DeleteTablet for tablet a8cb6b4b86fb44278ecf895c461bceaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:04 UTC) from {username='slave'} at 127.0.0.1:56458
I20260806 01:01:05.003654 12587 tablet_service.cc:1514] Processing DeleteTablet for tablet a8cb6b4b86fb44278ecf895c461bceaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:04 UTC) from {username='slave'} at 127.0.0.1:44682
I20260806 01:01:05.005605 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 12495
I20260806 01:01:05.016026 13107 tablet_replica.cc:330] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611: stopping tablet replica
I20260806 01:01:05.016655 13107 raft_consensus.cc:2229] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:05.017309 13107 raft_consensus.cc:2258] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:05.018214 13108 tablet_replica.cc:330] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: stopping tablet replica
I20260806 01:01:05.018729 13108 raft_consensus.cc:2229] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:01:05.019011 13106 tablet_replica.cc:330] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: stopping tablet replica
I20260806 01:01:05.019425 13106 raft_consensus.cc:2229] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:05.019508 13108 raft_consensus.cc:2258] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:05.020319 13106 raft_consensus.cc:2258] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:05.021023 13107 ts_tablet_manager.cc:1905] T a8cb6b4b86fb44278ecf895c461bceaa P 921fe885be854231b41f0c0b3d853611: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:05.021574 13108 ts_tablet_manager.cc:1905] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:05.021948 13071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.007083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.5.108.129:44559)
0806 01:01:05.007253 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:05.007258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:05.018396 (+ 11138us) server_negotiation.cc:407] Connection header received
0806 01:01:05.018911 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:05.018917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:05.019010 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:05.019111 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:05.021141 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:05.021147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:05.021152 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:05.021172 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:05.021212 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:05.021230 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:05.021353 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:05.021588 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 01:01:05.021652 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:01:05.023006 12672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:05.024329 13106 ts_tablet_manager.cc:1905] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:05.029281 13108 ts_tablet_manager.cc:1918] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:05.029567 13108 log.cc:1196] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/a8cb6b4b86fb44278ecf895c461bceaa
I20260806 01:01:05.030721 13108 ts_tablet_manager.cc:1939] T a8cb6b4b86fb44278ecf895c461bceaa P a58a7ac42b344525847da93c640a1eae: Deleting consensus metadata
I20260806 01:01:05.031102 13106 ts_tablet_manager.cc:1918] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:05.031296 13106 log.cc:1196] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/a8cb6b4b86fb44278ecf895c461bceaa
I20260806 01:01:05.031870 13106 ts_tablet_manager.cc:1939] T a8cb6b4b86fb44278ecf895c461bceaa P f03a4e9a009d404a98627b333ac0aae3: Deleting consensus metadata
I20260806 01:01:05.033275 12395 catalog_manager.cc:4841] TS a58a7ac42b344525847da93c640a1eae (127.5.108.131:34521): tablet a8cb6b4b86fb44278ecf895c461bceaa (table test-table [id=11d19b05de0a49d0b3a9b3d8e927fd3a]) successfully deleted
I20260806 01:01:05.033632 12395 catalog_manager.cc:4841] TS f03a4e9a009d404a98627b333ac0aae3 (127.5.108.130:44979): tablet a8cb6b4b86fb44278ecf895c461bceaa (table test-table [id=11d19b05de0a49d0b3a9b3d8e927fd3a]) successfully deleted
I20260806 01:01:05.115027 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 12495
W20260806 01:01:05.140596 12395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.108.129:44559 (error 108)
I20260806 01:01:05.141287 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 12686
I20260806 01:01:05.142664 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.142113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44559 (local address 127.0.0.1:44698)
0806 01:01:05.142344 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:05.142490 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:44559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 01:01:05.142908 12395 catalog_manager.cc:4584] TS 921fe885be854231b41f0c0b3d853611 (127.5.108.129:44559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8cb6b4b86fb44278ecf895c461bceaa: Network error: Client connection negotiation failed: client connection to 127.5.108.129:44559: connect: Connection refused (error 111)
I20260806 01:01:05.144574 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.142113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47286 (local address 127.5.108.130:44979)
0806 01:01:05.142679 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:05.142684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:05.142697 (+ 13us) server_negotiation.cc:407] Connection header received
0806 01:01:05.142785 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:05.142791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:05.142913 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:05.143034 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:05.144008 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:05.144013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:05.144016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:05.144033 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:05.144064 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:05.144076 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:05.144180 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:05.144313 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 01:01:05.144371 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461}
I20260806 01:01:05.145268 12857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:05.196869 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.196329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44559 (local address 127.0.0.1:44712)
0806 01:01:05.196567 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:05.196677 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:44559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 01:01:05.198055 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.197655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44559 (local address 127.0.0.1:44728)
0806 01:01:05.197820 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:05.197918 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:44559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 01:01:05.231401 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 12686
I20260806 01:01:05.256840 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 12869
I20260806 01:01:05.258678 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.258185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44559 (local address 127.0.0.1:44734)
0806 01:01:05.258366 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:05.258478 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:44559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 01:01:05.260223 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.259354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44559 (local address 127.0.0.1:44736)
0806 01:01:05.259967 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:05.260062 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:44559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 01:01:05.261058 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.258089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.5.108.131:34521)
0806 01:01:05.258300 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:05.258306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:05.258327 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:01:05.258412 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:05.258418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:05.258489 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:05.258632 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:05.260045 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:05.260054 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:05.260061 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:05.260113 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:05.260193 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:05.260215 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:05.260460 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:05.260689 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 01:01:05.260743 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:01:05.261840 13037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:05.340868 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.340315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44559 (local address 127.0.0.1:44748)
0806 01:01:05.340551 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:05.340666 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:44559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 01:01:05.342188 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:05.341559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44559 (local address 127.0.0.1:44752)
0806 01:01:05.341925 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:05.342009 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:44559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":18}
I20260806 01:01:05.381428 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 12869
I20260806 01:01:05.408109 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 12370
I20260806 01:01:05.409189 12453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:05.514775 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 12370
2026-08-06T01:01:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2591 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 01:01:05.563243 5554 mini_oidc.cc:152] Starting JWKS server
I20260806 01:01:05.566308 5554 webserver.cc:460] Webserver started at https://127.0.0.1:33343/ using document root <none> and password file <none>
I20260806 01:01:05.566606 5554 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 01:01:05.567206 5554 webserver.cc:460] Webserver started at http://127.0.0.1:41539/ using document root <none> and password file <none>
2026-08-06T01:01:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:05Z Disabled control of system clock
I20260806 01:01:05.599527 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:33689
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:43971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:33689
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33343/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/testchainca.cert with env {}
W20260806 01:01:05.936779 13123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:05.937111 13123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:05.937259 13123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:05.944900 13123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:05.945039 13123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:05.945120 13123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:05.945248 13123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:05.945333 13123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:01:05.945431 13123 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:01:05.945519 13123 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33343/jwks.json
I20260806 01:01:05.956571 13123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:43971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:33689
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:33689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33343/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:05.958461 13123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:05.959918 13123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:05.971984 13129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:05.972019 13131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:05.972080 13128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:05.973212 13123 server_base.cc:1034] running on GCE node
I20260806 01:01:05.974087 13123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:05.975673 13123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:05.976920 13123 hybrid_clock.cc:648] HybridClock initialized: now 1785978065976871 us; error 55 us; skew 500 ppm
I20260806 01:01:05.977316 13123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:05.979720 13123 webserver.cc:460] Webserver started at http://127.5.108.190:34341/ using document root <none> and password file <none>
I20260806 01:01:05.980437 13123 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:05.980593 13123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:05.980921 13123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:05.984040 13123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "0a5c56fda6d049a88defb25118e8124b"
format_stamp: "Formatted at 2026-08-06 01:01:05 on dist-test-slave-x4qh"
server_key: "f7b9cfd9d50d8a1f85ca4f0581b18ff8"
server_key_iv: "0408b6cacfce24d072985938630f3283"
server_key_version: "encryptionkey@0"
I20260806 01:01:05.984912 13123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "0a5c56fda6d049a88defb25118e8124b"
format_stamp: "Formatted at 2026-08-06 01:01:05 on dist-test-slave-x4qh"
server_key: "f7b9cfd9d50d8a1f85ca4f0581b18ff8"
server_key_iv: "0408b6cacfce24d072985938630f3283"
server_key_version: "encryptionkey@0"
I20260806 01:01:05.990108 13123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:01:05.993647 13137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:05.994890 13123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:01:05.995101 13123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "0a5c56fda6d049a88defb25118e8124b"
format_stamp: "Formatted at 2026-08-06 01:01:05 on dist-test-slave-x4qh"
server_key: "f7b9cfd9d50d8a1f85ca4f0581b18ff8"
server_key_iv: "0408b6cacfce24d072985938630f3283"
server_key_version: "encryptionkey@0"
I20260806 01:01:05.995342 13123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:06.011787 13123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:06.012603 13123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:06.012938 13123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:06.032055 13123 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:33689
I20260806 01:01:06.032081 13213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:33689 every 8 connection(s)
I20260806 01:01:06.033231 13123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:06.036774 13214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:06.038691 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 13123
I20260806 01:01:06.038908 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:06.039585 5554 external_mini_cluster.cc:1428] Setting key dd93e5f3ff27a035afe0652fab9ba5d2
I20260806 01:01:06.050283 13214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b: Bootstrap starting.
I20260806 01:01:06.054167 13214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:06.055202 13214 log.cc:824] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:06.058686 13214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b: No bootstrap required, opened a new log
I20260806 01:01:06.059324 13221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:06.041333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.5.108.190:33689)
0806 01:01:06.042340 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:06.042360 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:06.042415 (+ 55us) server_negotiation.cc:407] Connection header received
0806 01:01:06.045451 (+ 3036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:06.045478 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:06.046055 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:06.046475 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:06.047016 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.049152 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.050534 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.051680 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.051865 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:06.056547 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:06.056555 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:06.056568 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:06.057323 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:06.057425 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:06.057459 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:06.057575 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:06.057811 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 01:01:06.058108 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":304,"threads_started":1}
I20260806 01:01:06.063037 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:01:06.068210 13214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c56fda6d049a88defb25118e8124b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33689 } }
I20260806 01:01:06.068558 13214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:06.068682 13214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5c56fda6d049a88defb25118e8124b, State: Initialized, Role: FOLLOWER
I20260806 01:01:06.069227 13214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c56fda6d049a88defb25118e8124b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33689 } }
I20260806 01:01:06.069432 13214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:06.069536 13214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:06.069672 13214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:06.071974 13214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c56fda6d049a88defb25118e8124b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33689 } }
I20260806 01:01:06.072640 13214 leader_election.cc:302] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a5c56fda6d049a88defb25118e8124b; no voters:
I20260806 01:01:06.073303 13214 leader_election.cc:288] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:06.073479 13223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:06.074311 13223 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [term 1 LEADER]: Becoming Leader. State: Replica: 0a5c56fda6d049a88defb25118e8124b, State: Running, Role: LEADER
I20260806 01:01:06.074867 13223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c56fda6d049a88defb25118e8124b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33689 } }
I20260806 01:01:06.075593 13214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:06.078099 13224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a5c56fda6d049a88defb25118e8124b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c56fda6d049a88defb25118e8124b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33689 } } }
I20260806 01:01:06.078614 13224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:06.079155 13225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a5c56fda6d049a88defb25118e8124b. Latest consensus state: current_term: 1 leader_uuid: "0a5c56fda6d049a88defb25118e8124b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5c56fda6d049a88defb25118e8124b" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 33689 } } }
I20260806 01:01:06.079526 13225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:06.083123 13236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:06.087911 13236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:06.095743 13236 catalog_manager.cc:1347] Generated new cluster ID: 40b6a64052b848ce99cdcd77663421c1
I20260806 01:01:06.095832 13236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:06.110844 13236 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:06.112427 13236 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:06.133167 13236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a5c56fda6d049a88defb25118e8124b: Generated new TSK 0
I20260806 01:01:06.133821 13236 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 01:01:06.179829 13258 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 01:01:06.180063 13258 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 01:01:06.180954 13221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:06.158773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54136 (local address 127.5.108.190:33689)
0806 01:01:06.158958 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:06.158964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:06.159526 (+ 562us) server_negotiation.cc:407] Connection header received
0806 01:01:06.159836 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:06.159854 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:06.159939 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:06.160086 (+ 147us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:01:06.160553 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.161694 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.163055 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.163705 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.163823 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:06.164330 (+ 507us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:01:06.180394 (+ 16064us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 01:01:06.180684 (+ 290us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54136: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 01:01:06.181059 13221 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54136: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 01:01:06.182243 13257 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:06.158707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:33689 (local address 127.0.0.1:54136)
0806 01:01:06.159428 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:06.159451 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:06.159593 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:06.160115 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:06.160119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:06.160127 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:01:06.160417 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:06.160423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.161848 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.161852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:06.162918 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:06.162928 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.163856 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.163860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:06.163968 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:06.164036 (+ 68us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:01:06.180600 (+ 16564us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 01:01:06.180809 (+ 209us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.108.190:33689: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":338,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.108.190:33689: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 01:01:06.183787 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 13123
I20260806 01:01:06.184875 13209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:06.274794 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 13123
2026-08-06T01:01:06Z chronyd exiting
I20260806 01:01:06.334147 5554 test_util.cc:175] -----------------------------------------------
I20260806 01:01:06.334311 5554 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978022745104-5554-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (775 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 01:01:06.338034 5554 mini_oidc.cc:152] Starting JWKS server
I20260806 01:01:06.339789 5554 webserver.cc:460] Webserver started at https://127.0.0.1:43587/ using document root <none> and password file <none>
I20260806 01:01:06.340018 5554 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 01:01:06.340602 5554 webserver.cc:460] Webserver started at http://127.0.0.1:33869/ using document root <none> and password file <none>
2026-08-06T01:01:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:06Z Disabled control of system clock
I20260806 01:01:06.373350 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:32851
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:38743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:32851
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43587/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/testchainca.cert with env {}
W20260806 01:01:06.692039 13267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:06.692380 13267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:06.692548 13267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:06.699604 13267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:06.699738 13267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:06.699836 13267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:06.699927 13267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:06.700007 13267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:01:06.700093 13267 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:01:06.700163 13267 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43587/jwks.json
I20260806 01:01:06.709323 13267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:32851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:32851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43587/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:06.710546 13267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:06.711812 13267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:06.723434 13276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:06.723634 13274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:06.723591 13278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:06.724476 13267 server_base.cc:1034] running on GCE node
I20260806 01:01:06.725158 13267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:06.726468 13267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:06.727660 13267 hybrid_clock.cc:648] HybridClock initialized: now 1785978066727605 us; error 53 us; skew 500 ppm
I20260806 01:01:06.728072 13267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:06.730475 13267 webserver.cc:460] Webserver started at http://127.5.108.190:34947/ using document root <none> and password file <none>
I20260806 01:01:06.731204 13267 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:06.731343 13267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:06.731698 13267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:06.734632 13267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "6f2287080e89431d8986642e39eb1cf0"
format_stamp: "Formatted at 2026-08-06 01:01:06 on dist-test-slave-x4qh"
server_key: "c6ec236ce7b1bf8d33bc973b8bc9e0da"
server_key_iv: "1198f48dc67e7971d83bcac407f7afdb"
server_key_version: "encryptionkey@0"
I20260806 01:01:06.735505 13267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "6f2287080e89431d8986642e39eb1cf0"
format_stamp: "Formatted at 2026-08-06 01:01:06 on dist-test-slave-x4qh"
server_key: "c6ec236ce7b1bf8d33bc973b8bc9e0da"
server_key_iv: "1198f48dc67e7971d83bcac407f7afdb"
server_key_version: "encryptionkey@0"
I20260806 01:01:06.740335 13267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 01:01:06.743697 13286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:06.745046 13267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:01:06.745250 13267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "6f2287080e89431d8986642e39eb1cf0"
format_stamp: "Formatted at 2026-08-06 01:01:06 on dist-test-slave-x4qh"
server_key: "c6ec236ce7b1bf8d33bc973b8bc9e0da"
server_key_iv: "1198f48dc67e7971d83bcac407f7afdb"
server_key_version: "encryptionkey@0"
I20260806 01:01:06.745486 13267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:06.762818 13267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:06.763711 13267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:06.764015 13267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:06.782173 13267 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:32851
I20260806 01:01:06.782195 13368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:32851 every 8 connection(s)
I20260806 01:01:06.783427 13267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:06.786943 13369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:06.791780 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 13267
I20260806 01:01:06.791972 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:06.792616 5554 external_mini_cluster.cc:1428] Setting key ecc60946cd9b95a71996bd11a1e3caf0
I20260806 01:01:06.797192 13369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0: Bootstrap starting.
I20260806 01:01:06.800700 13369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:06.801890 13369 log.cc:824] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:06.804996 13369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0: No bootstrap required, opened a new log
I20260806 01:01:06.806169 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:06.794543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59856 (local address 127.5.108.190:32851)
0806 01:01:06.795685 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:06.795703 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:06.795743 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:01:06.797322 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:06.797355 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:06.797822 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:06.798101 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:06.798695 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.799968 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.801823 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.802706 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.802908 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:06.804004 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:06.804012 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:06.804026 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:06.804539 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:06.804610 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:06.804631 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:06.804724 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:06.804965 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 01:01:06.805268 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":419,"threads_started":1}
I20260806 01:01:06.810269 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:01:06.813712 13369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2287080e89431d8986642e39eb1cf0" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32851 } }
I20260806 01:01:06.814039 13369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:06.814167 13369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2287080e89431d8986642e39eb1cf0, State: Initialized, Role: FOLLOWER
I20260806 01:01:06.814685 13369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2287080e89431d8986642e39eb1cf0" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32851 } }
I20260806 01:01:06.814919 13369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:06.815065 13369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:06.815241 13369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:06.817348 13369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2287080e89431d8986642e39eb1cf0" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32851 } }
I20260806 01:01:06.817827 13369 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f2287080e89431d8986642e39eb1cf0; no voters:
I20260806 01:01:06.818534 13369 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:06.818666 13376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:06.819507 13376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [term 1 LEADER]: Becoming Leader. State: Replica: 6f2287080e89431d8986642e39eb1cf0, State: Running, Role: LEADER
I20260806 01:01:06.819990 13376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2287080e89431d8986642e39eb1cf0" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32851 } }
I20260806 01:01:06.820676 13369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:06.822813 13378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2287080e89431d8986642e39eb1cf0. Latest consensus state: current_term: 1 leader_uuid: "6f2287080e89431d8986642e39eb1cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2287080e89431d8986642e39eb1cf0" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32851 } } }
I20260806 01:01:06.823081 13377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f2287080e89431d8986642e39eb1cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2287080e89431d8986642e39eb1cf0" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32851 } } }
I20260806 01:01:06.823273 13378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:06.823397 13377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:06.825161 13390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:06.829408 13390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:06.837216 13390 catalog_manager.cc:1347] Generated new cluster ID: d72d1b3bb18a4d589dc9e14d11e6cebc
I20260806 01:01:06.837316 13390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:06.887285 13390 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:06.888943 13390 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:06.896499 13390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f2287080e89431d8986642e39eb1cf0: Generated new TSK 0
I20260806 01:01:06.897454 13390 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 01:01:06.938201 13419 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 01:01:06.938316 13419 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 01:01:06.939079 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:06.912996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59868 (local address 127.5.108.190:32851)
0806 01:01:06.913245 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:06.913253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:06.913627 (+ 374us) server_negotiation.cc:407] Connection header received
0806 01:01:06.913824 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:06.913827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:06.913897 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:06.914022 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:01:06.914590 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.915315 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.917329 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.917948 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.918053 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:06.918613 (+ 560us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:01:06.938625 (+ 20012us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 01:01:06.938897 (+ 272us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59868: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 01:01:06.939232 13374 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59868: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 01:01:06.939566 13418 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:06.912876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:32851 (local address 127.0.0.1:59868)
0806 01:01:06.913516 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:06.913540 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:06.913682 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:06.914086 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:06.914092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:06.914105 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:01:06.914420 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:06.914433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.915471 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.915477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:06.917185 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:06.917200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:06.918144 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:06.918151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:06.918346 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:06.918391 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:01:06.938824 (+ 20433us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 01:01:06.939052 (+ 228us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.108.190:32851: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":367,"threads_started":1}
I20260806 01:01:06.940877 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 13267
I20260806 01:01:06.941933 13363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:07.034574 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 13267
2026-08-06T01:01:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (759 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 01:01:07.098078 5554 mini_oidc.cc:152] Starting JWKS server
I20260806 01:01:07.099941 5554 webserver.cc:460] Webserver started at https://127.0.0.1:44395/ using document root <none> and password file <none>
I20260806 01:01:07.100158 5554 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 01:01:07.100785 5554 webserver.cc:460] Webserver started at http://127.0.0.1:40211/ using document root <none> and password file <none>
2026-08-06T01:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:07Z Disabled control of system clock
I20260806 01:01:07.134593 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34809
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:41271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:34809
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44395/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 01:01:07.449797 13430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:07.450070 13430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:07.450228 13430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:07.457350 13430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:07.457435 13430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:07.457470 13430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:07.457561 13430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:07.457625 13430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:01:07.457762 13430 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:01:07.457819 13430 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44395/jwks.json
I20260806 01:01:07.466523 13430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:41271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:34809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44395/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:07.467746 13430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:07.469496 13430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:07.480805 13437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:07.480912 13436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:07.481305 13441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:07.481904 13430 server_base.cc:1034] running on GCE node
I20260806 01:01:07.482606 13430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:07.484051 13430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:07.485316 13430 hybrid_clock.cc:648] HybridClock initialized: now 1785978067485268 us; error 56 us; skew 500 ppm
I20260806 01:01:07.485671 13430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:07.487833 13430 webserver.cc:460] Webserver started at http://127.5.108.190:41515/ using document root <none> and password file <none>
I20260806 01:01:07.488463 13430 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:07.488600 13430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:07.488956 13430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:07.491834 13430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7"
format_stamp: "Formatted at 2026-08-06 01:01:07 on dist-test-slave-x4qh"
server_key: "575e94ada1b2123b26bfc4db76043c22"
server_key_iv: "c79f3626b155db622442a68a2478d88f"
server_key_version: "encryptionkey@0"
I20260806 01:01:07.492614 13430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7"
format_stamp: "Formatted at 2026-08-06 01:01:07 on dist-test-slave-x4qh"
server_key: "575e94ada1b2123b26bfc4db76043c22"
server_key_iv: "c79f3626b155db622442a68a2478d88f"
server_key_version: "encryptionkey@0"
I20260806 01:01:07.497491 13430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 01:01:07.500659 13450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:07.501860 13430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 01:01:07.502038 13430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7"
format_stamp: "Formatted at 2026-08-06 01:01:07 on dist-test-slave-x4qh"
server_key: "575e94ada1b2123b26bfc4db76043c22"
server_key_iv: "c79f3626b155db622442a68a2478d88f"
server_key_version: "encryptionkey@0"
I20260806 01:01:07.502245 13430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:07.520063 13430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:07.520777 13430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:07.521083 13430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:07.538916 13430 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:34809
I20260806 01:01:07.538934 13527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:34809 every 8 connection(s)
I20260806 01:01:07.540287 13430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:07.543864 13528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:07.544379 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 13430
I20260806 01:01:07.544602 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:07.545251 5554 external_mini_cluster.cc:1428] Setting key 7d74be878b9838110c95eef15c2e1608
I20260806 01:01:07.558245 13528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7: Bootstrap starting.
I20260806 01:01:07.560250 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:07.547290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.5.108.190:34809)
0806 01:01:07.548364 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:07.548383 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:07.548433 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:01:07.550557 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:07.550584 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:07.551153 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:07.551617 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:07.552123 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:07.553946 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:07.555258 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:07.556397 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:07.556559 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:07.557591 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:07.557601 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:07.557616 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:07.558269 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:07.558366 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:07.558391 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:07.558489 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:07.558691 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 01:01:07.558968 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":319,"threads_started":1}
I20260806 01:01:07.561729 13528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:07.562829 13528 log.cc:824] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:07.563952 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:01:07.566035 13528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7: No bootstrap required, opened a new log
I20260806 01:01:07.573633 13528 raft_consensus.cc:348] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34809 } }
I20260806 01:01:07.573937 13528 raft_consensus.cc:374] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:07.574007 13528 raft_consensus.cc:729] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b688e3f1a75d4045bf1fd060bfd8cfa7, State: Initialized, Role: FOLLOWER
I20260806 01:01:07.574468 13528 consensus_queue.cc:260] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34809 } }
I20260806 01:01:07.574662 13528 raft_consensus.cc:388] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:07.574764 13528 raft_consensus.cc:482] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:07.574877 13528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:07.576894 13528 raft_consensus.cc:504] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34809 } }
I20260806 01:01:07.577308 13528 leader_election.cc:302] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b688e3f1a75d4045bf1fd060bfd8cfa7; no voters:
I20260806 01:01:07.578007 13528 leader_election.cc:288] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:07.578191 13534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:07.578961 13534 raft_consensus.cc:686] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [term 1 LEADER]: Becoming Leader. State: Replica: b688e3f1a75d4045bf1fd060bfd8cfa7, State: Running, Role: LEADER
I20260806 01:01:07.579483 13534 consensus_queue.cc:237] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34809 } }
I20260806 01:01:07.580101 13528 sys_catalog.cc:564] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:07.582551 13535 sys_catalog.cc:455] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34809 } } }
I20260806 01:01:07.582754 13536 sys_catalog.cc:455] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b688e3f1a75d4045bf1fd060bfd8cfa7. Latest consensus state: current_term: 1 leader_uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b688e3f1a75d4045bf1fd060bfd8cfa7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34809 } } }
I20260806 01:01:07.583024 13536 sys_catalog.cc:458] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:07.583024 13535 sys_catalog.cc:458] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:07.584632 13545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:07.589536 13545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:07.596944 13545 catalog_manager.cc:1347] Generated new cluster ID: 3e2f3e552a8f483184e269ab2f232b81
I20260806 01:01:07.597085 13545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:07.617935 13545 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:07.619202 13545 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:07.631039 13545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b688e3f1a75d4045bf1fd060bfd8cfa7: Generated new TSK 0
I20260806 01:01:07.632004 13545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:01:07.658452 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:07.650107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.5.108.190:34809)
0806 01:01:07.650364 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:07.650371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:07.650772 (+ 401us) server_negotiation.cc:407] Connection header received
0806 01:01:07.651006 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:07.651009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:07.651085 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:07.651205 (+ 120us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:01:07.651814 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:07.652525 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:07.654606 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:07.655082 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:07.655171 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:07.655839 (+ 668us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:01:07.657777 (+ 1938us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 01:01:07.658134 (+ 357us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 01:01:07.658643 13571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:07.649990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34809 (local address 127.0.0.1:37602)
0806 01:01:07.650656 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:07.650676 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:07.650831 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:07.651274 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:07.651282 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:07.651297 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:01:07.651643 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:07.651656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:07.652684 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:07.652691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:07.654458 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:07.654474 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:07.655290 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:07.655297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:07.655530 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:07.655577 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:01:07.657977 (+ 2400us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 01:01:07.658222 (+ 245us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.108.190:34809: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":346,"threads_started":1}
W20260806 01:01:07.659093 13532 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260806 01:01:07.660220 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 13430
I20260806 01:01:07.661306 13521 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 01:01:07.670850 13573 webserver.cc:533] Webserver: sslize failed:
E20260806 01:01:07.670953 13573 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 01:01:07.751804 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 13430
2026-08-06T01:01:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (714 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T01:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:07Z Disabled control of system clock
I20260806 01:01:07.845513 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37889
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:38257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:37889
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/testchainca.cert with env {}
W20260806 01:01:08.163985 13580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:08.164286 13580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:08.164482 13580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:08.172020 13580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:08.172108 13580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:08.172200 13580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:08.172282 13580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:08.172364 13580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:01:08.172472 13580 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:01:08.172550 13580 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 01:01:08.181758 13580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:37889
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:08.182936 13580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:08.184139 13580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:08.195251 13590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:08.195339 13588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:08.195312 13593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:08.196485 13580 server_base.cc:1034] running on GCE node
I20260806 01:01:08.197214 13580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:08.198537 13580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:08.199862 13580 hybrid_clock.cc:648] HybridClock initialized: now 1785978068199827 us; error 53 us; skew 500 ppm
I20260806 01:01:08.200222 13580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:08.202315 13580 webserver.cc:460] Webserver started at http://127.5.108.190:44211/ using document root <none> and password file <none>
I20260806 01:01:08.202931 13580 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:08.203058 13580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:08.203367 13580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:08.206037 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "28efdfa03c5f4d4b9227513aaea8227d"
format_stamp: "Formatted at 2026-08-06 01:01:08 on dist-test-slave-x4qh"
server_key: "19fd82414c7907889f3669538cffc218"
server_key_iv: "53ea1e90fef8d91eba794952dd9dc46b"
server_key_version: "encryptionkey@0"
I20260806 01:01:08.206779 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "28efdfa03c5f4d4b9227513aaea8227d"
format_stamp: "Formatted at 2026-08-06 01:01:08 on dist-test-slave-x4qh"
server_key: "19fd82414c7907889f3669538cffc218"
server_key_iv: "53ea1e90fef8d91eba794952dd9dc46b"
server_key_version: "encryptionkey@0"
I20260806 01:01:08.211622 13580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:01:08.214747 13600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:08.216009 13580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:08.216192 13580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "28efdfa03c5f4d4b9227513aaea8227d"
format_stamp: "Formatted at 2026-08-06 01:01:08 on dist-test-slave-x4qh"
server_key: "19fd82414c7907889f3669538cffc218"
server_key_iv: "53ea1e90fef8d91eba794952dd9dc46b"
server_key_version: "encryptionkey@0"
I20260806 01:01:08.216413 13580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:08.249383 13580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:08.250106 13580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:08.250387 13580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:08.268339 13580 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:37889
I20260806 01:01:08.268373 13683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:37889 every 8 connection(s)
I20260806 01:01:08.269547 13580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:08.272980 13684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:08.274454 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 13580
I20260806 01:01:08.274659 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:08.275338 5554 external_mini_cluster.cc:1428] Setting key 33d7a86b66532da2b51c4379a6d5e832
I20260806 01:01:08.283859 13684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d: Bootstrap starting.
I20260806 01:01:08.287261 13684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:08.288121 13688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:08.277111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34986 (local address 127.5.108.190:37889)
0806 01:01:08.278119 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:08.278129 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:08.278167 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:01:08.279818 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:08.279838 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:08.280204 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:08.280473 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:08.281048 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:08.282206 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:08.284040 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:08.284715 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:08.284846 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:08.286060 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:08.286066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:08.286076 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:08.286544 (+ 468us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:08.286625 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:08.286643 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:08.286725 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:08.286932 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 01:01:08.287216 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":350,"threads_started":1}
I20260806 01:01:08.288276 13684 log.cc:824] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:08.291589 13684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d: No bootstrap required, opened a new log
I20260806 01:01:08.292207 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:01:08.299630 13684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28efdfa03c5f4d4b9227513aaea8227d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37889 } }
I20260806 01:01:08.299961 13684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:08.300071 13684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28efdfa03c5f4d4b9227513aaea8227d, State: Initialized, Role: FOLLOWER
I20260806 01:01:08.300637 13684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28efdfa03c5f4d4b9227513aaea8227d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37889 } }
I20260806 01:01:08.300887 13684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:08.301048 13684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:08.301244 13684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:08.303213 13684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28efdfa03c5f4d4b9227513aaea8227d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37889 } }
I20260806 01:01:08.303747 13684 leader_election.cc:302] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28efdfa03c5f4d4b9227513aaea8227d; no voters:
I20260806 01:01:08.304306 13684 leader_election.cc:288] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:08.304417 13690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:08.305181 13690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [term 1 LEADER]: Becoming Leader. State: Replica: 28efdfa03c5f4d4b9227513aaea8227d, State: Running, Role: LEADER
I20260806 01:01:08.305653 13690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28efdfa03c5f4d4b9227513aaea8227d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37889 } }
I20260806 01:01:08.306367 13684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:08.308454 13691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28efdfa03c5f4d4b9227513aaea8227d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28efdfa03c5f4d4b9227513aaea8227d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37889 } } }
I20260806 01:01:08.308528 13692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28efdfa03c5f4d4b9227513aaea8227d. Latest consensus state: current_term: 1 leader_uuid: "28efdfa03c5f4d4b9227513aaea8227d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28efdfa03c5f4d4b9227513aaea8227d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37889 } } }
I20260806 01:01:08.308794 13691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:08.308853 13692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:08.312721 13696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:08.317102 13696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:08.325156 13696 catalog_manager.cc:1347] Generated new cluster ID: 0f0ab5ab62304111bdd614adb9704504
I20260806 01:01:08.325281 13696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:08.340705 13696 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:08.341958 13696 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:08.356638 13696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28efdfa03c5f4d4b9227513aaea8227d: Generated new TSK 0
I20260806 01:01:08.357355 13696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:01:08.506494 13688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:08.377009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35000 (local address 127.5.108.190:37889)
0806 01:01:08.377201 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:08.377207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:08.377580 (+ 373us) server_negotiation.cc:407] Connection header received
0806 01:01:08.377767 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:08.377773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:08.377868 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:08.378031 (+ 163us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:01:08.378455 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:08.379617 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:08.380962 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:08.381637 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:08.381769 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:08.382170 (+ 401us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:01:08.505683 (+123513us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 01:01:08.506112 (+ 429us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35000: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 01:01:08.506639 13688 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35000: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 01:01:08.506726 13721 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:08.376856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37889 (local address 127.0.0.1:35000)
0806 01:01:08.377481 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:08.377496 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:08.377610 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:08.378045 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:08.378049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:08.378058 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:01:08.378318 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:08.378325 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:08.379768 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:08.379772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:08.380798 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:08.380808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:08.381803 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:08.381808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:08.381915 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:08.381943 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:01:08.505972 (+124029us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 01:01:08.506286 (+ 314us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.108.190:37889: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":339,"threads_started":1}
I20260806 01:01:08.508289 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 13580
I20260806 01:01:08.509480 13672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:08.605952 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 13580
2026-08-06T01:01:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (837 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (404 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (395 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:01:09 dist-test-slave-x4qh krb5kdc[13747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:01:09 dist-test-slave-x4qh krb5kdc[13747](info): set up 2 sockets
Aug 06 01:01:09 dist-test-slave-x4qh krb5kdc[13747](info): commencing operation
krb5kdc: starting...
W20260806 01:01:10.639323 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:01:10 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:01:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:01:10.852655 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:46803
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:46803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:11.176528 13766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:11.176802 13766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:11.176891 13766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:11.183913 13766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:11.184005 13766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:11.184058 13766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:11.184114 13766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:11.184149 13766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:11.193646 13766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:46803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:46803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:11.194778 13766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:11.196025 13766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:11.207216 13775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:11.207258 13777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:11.207175 13779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:11.208545 13766 server_base.cc:1034] running on GCE node
I20260806 01:01:11.209192 13766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:11.210440 13766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:11.211654 13766 hybrid_clock.cc:648] HybridClock initialized: now 1785978071211560 us; error 66 us; skew 500 ppm
I20260806 01:01:11.211948 13766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:11 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978071, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:11.218863 13766 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:01:11.220296 13766 webserver.cc:460] Webserver started at http://127.5.108.190:36365/ using document root <none> and password file <none>
I20260806 01:01:11.220899 13766 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:11.221052 13766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:11.221361 13766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:11.224066 13766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "a901eb604b0c4d918d599268c60dc1b5"
format_stamp: "Formatted at 2026-08-06 01:01:11 on dist-test-slave-x4qh"
server_key: "410c749fd649a8041229f35432925d7c"
server_key_iv: "4e938ca73e42b64d6903fc3f2886ddfb"
server_key_version: "encryptionkey@0"
I20260806 01:01:11.224884 13766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "a901eb604b0c4d918d599268c60dc1b5"
format_stamp: "Formatted at 2026-08-06 01:01:11 on dist-test-slave-x4qh"
server_key: "410c749fd649a8041229f35432925d7c"
server_key_iv: "4e938ca73e42b64d6903fc3f2886ddfb"
server_key_version: "encryptionkey@0"
I20260806 01:01:11.229655 13766 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 01:01:11.232997 13790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:11.234197 13766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:01:11.234387 13766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "a901eb604b0c4d918d599268c60dc1b5"
format_stamp: "Formatted at 2026-08-06 01:01:11 on dist-test-slave-x4qh"
server_key: "410c749fd649a8041229f35432925d7c"
server_key_iv: "4e938ca73e42b64d6903fc3f2886ddfb"
server_key_version: "encryptionkey@0"
I20260806 01:01:11.234596 13766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:11.248715 13766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:11.252493 13766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:11.252765 13766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:11.270599 13766 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:46803
I20260806 01:01:11.270685 13872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:46803 every 8 connection(s)
I20260806 01:01:11.271871 13766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:11.275342 13876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:11.282253 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 13766
I20260806 01:01:11.282473 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:11.283077 5554 external_mini_cluster.cc:1428] Setting key 6b265eb5fc63822e3803d97e18b87756
I20260806 01:01:11.285434 13876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5: Bootstrap starting.
I20260806 01:01:11.288715 13876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:11.289981 13876 log.cc:824] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:11.293195 13876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5: No bootstrap required, opened a new log
Aug 06 01:01:11 dist-test-slave-x4qh krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:11.301807 13876 raft_consensus.cc:348] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a901eb604b0c4d918d599268c60dc1b5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46803 } }
I20260806 01:01:11.302196 13876 raft_consensus.cc:374] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:11.302325 13876 raft_consensus.cc:729] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a901eb604b0c4d918d599268c60dc1b5, State: Initialized, Role: FOLLOWER
I20260806 01:01:11.302881 13876 consensus_queue.cc:260] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a901eb604b0c4d918d599268c60dc1b5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46803 } }
I20260806 01:01:11.303140 13876 raft_consensus.cc:388] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:11.303321 13876 raft_consensus.cc:482] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:11.303517 13876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:11.305728 13876 raft_consensus.cc:504] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a901eb604b0c4d918d599268c60dc1b5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46803 } }
I20260806 01:01:11.306286 13876 leader_election.cc:302] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a901eb604b0c4d918d599268c60dc1b5; no voters:
I20260806 01:01:11.306967 13876 leader_election.cc:288] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:11.307070 13884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:11.307256 13881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:11.285061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.5.108.190:46803)
0806 01:01:11.285910 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:11.285921 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:11.285957 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:01:11.287843 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:11.287864 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:11.288202 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:11.288488 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:11.290398 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:11.292245 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:11.294130 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:11.294798 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:11.294935 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:11.299557 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:11.299564 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:11.299578 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:11.299619 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:11.303009 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:11.303835 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:11.303844 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:11.303854 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:11.304033 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:11.304452 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:11.304457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:11.304461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:11.304994 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:11.305313 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:11.305742 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 01:01:11.306139 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":379,"threads_started":1}
I20260806 01:01:11.309448 13876 sys_catalog.cc:564] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:11.313855 13884 raft_consensus.cc:686] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [term 1 LEADER]: Becoming Leader. State: Replica: a901eb604b0c4d918d599268c60dc1b5, State: Running, Role: LEADER
I20260806 01:01:11.314426 13884 consensus_queue.cc:237] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a901eb604b0c4d918d599268c60dc1b5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46803 } }
I20260806 01:01:11.316917 13885 sys_catalog.cc:455] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a901eb604b0c4d918d599268c60dc1b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a901eb604b0c4d918d599268c60dc1b5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46803 } } }
I20260806 01:01:11.317175 13885 sys_catalog.cc:458] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:11.317848 13886 sys_catalog.cc:455] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a901eb604b0c4d918d599268c60dc1b5. Latest consensus state: current_term: 1 leader_uuid: "a901eb604b0c4d918d599268c60dc1b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a901eb604b0c4d918d599268c60dc1b5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46803 } } }
I20260806 01:01:11.318115 13886 sys_catalog.cc:458] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:11.318596 13903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:11.323202 13903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:11.331323 13903 catalog_manager.cc:1347] Generated new cluster ID: bf161e9bff0440dab4cdd45a0ac1ae63
I20260806 01:01:11.331459 13903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:11.353614 13903 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:11.354866 13903 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:11.361896 13903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a901eb604b0c4d918d599268c60dc1b5: Generated new TSK 0
I20260806 01:01:11.362697 13903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:01:11.424198 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46803
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:11.743474 13913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:11.743714 13913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:11.743871 13913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:11.750698 13913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:11.750880 13913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:11.751055 13913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:11.760083 13913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:11.761265 13913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:11.762456 13913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:11.775115 13927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:11.775092 13922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:11.775394 13921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:11.775717 13913 server_base.cc:1034] running on GCE node
I20260806 01:01:11.776505 13913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:11.777432 13913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:11.778687 13913 hybrid_clock.cc:648] HybridClock initialized: now 1785978071778641 us; error 49 us; skew 500 ppm
I20260806 01:01:11.779021 13913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:11 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978071, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:11.782121 13913 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:01:11.783524 13913 webserver.cc:460] Webserver started at http://127.5.108.129:35561/ using document root <none> and password file <none>
I20260806 01:01:11.784168 13913 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:11.784364 13913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:11.784677 13913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:11.787495 13913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "9ce901fd22f9490ba60350e820f1535f"
format_stamp: "Formatted at 2026-08-06 01:01:11 on dist-test-slave-x4qh"
server_key: "592ea650f4c26404f8b5f858d7629669"
server_key_iv: "377b1fe1fdfb1d48661e0aaaf1dc362b"
server_key_version: "encryptionkey@0"
I20260806 01:01:11.788257 13913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "9ce901fd22f9490ba60350e820f1535f"
format_stamp: "Formatted at 2026-08-06 01:01:11 on dist-test-slave-x4qh"
server_key: "592ea650f4c26404f8b5f858d7629669"
server_key_iv: "377b1fe1fdfb1d48661e0aaaf1dc362b"
server_key_version: "encryptionkey@0"
I20260806 01:01:11.793035 13913 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:01:11.796219 13938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:11.797353 13913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:01:11.797523 13913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "9ce901fd22f9490ba60350e820f1535f"
format_stamp: "Formatted at 2026-08-06 01:01:11 on dist-test-slave-x4qh"
server_key: "592ea650f4c26404f8b5f858d7629669"
server_key_iv: "377b1fe1fdfb1d48661e0aaaf1dc362b"
server_key_version: "encryptionkey@0"
I20260806 01:01:11.797714 13913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:11.816246 13913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:11.819943 13913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:11.820217 13913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:11.821096 13913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:11.822414 13913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:11.822487 13913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:11.822572 13913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:11.822619 13913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:11.854589 13913 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:35063
I20260806 01:01:11.854681 14115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:35063 every 8 connection(s)
I20260806 01:01:11.855794 13913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:11.862758 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 13913
I20260806 01:01:11.862972 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:01:11.863665 5554 external_mini_cluster.cc:1428] Setting key 73048c7adee84e2ed29fd272fd48bc43
Aug 06 01:01:11 dist-test-slave-x4qh krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978071, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:11.876201 14125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:11.858128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:44765 (local address 127.5.108.190:46803)
0806 01:01:11.858596 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:11.858600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:11.859314 (+ 714us) server_negotiation.cc:407] Connection header received
0806 01:01:11.861085 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:11.861106 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:11.861189 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:11.861304 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:11.862941 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:11.863856 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:11.865669 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:11.866072 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:11.866225 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:11.870125 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:11.870130 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:11.870133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:11.870171 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:11.872673 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:11.873587 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:11.873593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:11.873596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:11.873726 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:11.874201 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:11.874208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:11.874213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:11.874628 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:11.874782 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:11.875472 (+ 690us) server_negotiation.cc:299] Negotiation successful
0806 01:01:11.875725 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":265,"threads_started":1}
I20260806 01:01:11.876716 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:11.858498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46803 (local address 127.5.108.129:44765)
0806 01:01:11.859169 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:11.859202 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:11.860818 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:11.861557 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:11.861565 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:11.862024 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:11.862708 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:11.862737 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:11.864008 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:11.864015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:11.865519 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:11.865532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:11.866320 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:11.866328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:11.866554 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:11.867246 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:11.867272 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:11.869894 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:11.872808 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:11.872819 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:11.872835 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:11.873461 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:11.873828 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:11.873835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:11.873841 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:11.874067 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:11.874763 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:11.874769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:11.875051 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 01:01:11.875285 (+ 234us) client_negotiation.cc:240] Negotiation successful
0806 01:01:11.875687 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":288,"threads_started":1}
I20260806 01:01:11.878177 14121 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46803
I20260806 01:01:11.878475 14121 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:11.879416 14121 heartbeater.cc:511] Master 127.5.108.190:46803 requested a full tablet report, sending...
I20260806 01:01:11.881464 13814 ts_manager.cc:194] Registered new tserver with Master: 9ce901fd22f9490ba60350e820f1535f (127.5.108.129:35063)
I20260806 01:01:11.882855 13814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:44765
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:01:11.916095 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46803
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:12.233879 14130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:12.234167 14130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:12.234328 14130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:12.245901 14130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:12.246031 14130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:12.246187 14130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:12.255575 14130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:12.256850 14130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:12.258107 14130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:12.271831 14139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:12.272084 14140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:12.272755 14142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:12.273172 14130 server_base.cc:1034] running on GCE node
I20260806 01:01:12.273762 14130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:12.274830 14130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:12.276058 14130 hybrid_clock.cc:648] HybridClock initialized: now 1785978072275998 us; error 47 us; skew 500 ppm
I20260806 01:01:12.276355 14130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:12.279878 14130 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:01:12.281689 14130 webserver.cc:460] Webserver started at http://127.5.108.130:42357/ using document root <none> and password file <none>
I20260806 01:01:12.282572 14130 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:12.282712 14130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:12.283052 14130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:12.287329 14130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "0ad35a294f5046b7895324e7d686ecb3"
format_stamp: "Formatted at 2026-08-06 01:01:12 on dist-test-slave-x4qh"
server_key: "66433599ecc6974281f1c1903f3d3c28"
server_key_iv: "c7149f9d7846c2e9bcd24759bf61fe9e"
server_key_version: "encryptionkey@0"
I20260806 01:01:12.288394 14130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "0ad35a294f5046b7895324e7d686ecb3"
format_stamp: "Formatted at 2026-08-06 01:01:12 on dist-test-slave-x4qh"
server_key: "66433599ecc6974281f1c1903f3d3c28"
server_key_iv: "c7149f9d7846c2e9bcd24759bf61fe9e"
server_key_version: "encryptionkey@0"
I20260806 01:01:12.296227 14130 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 01:01:12.300990 14156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:12.302570 14130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:01:12.302783 14130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "0ad35a294f5046b7895324e7d686ecb3"
format_stamp: "Formatted at 2026-08-06 01:01:12 on dist-test-slave-x4qh"
server_key: "66433599ecc6974281f1c1903f3d3c28"
server_key_iv: "c7149f9d7846c2e9bcd24759bf61fe9e"
server_key_version: "encryptionkey@0"
I20260806 01:01:12.303040 14130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:12.326984 14130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:12.332711 14130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:12.333084 14130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:12.334450 14130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:12.336506 14130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:12.336621 14130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:12.336784 14130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:12.336874 14130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:12.375339 14130 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:33837
I20260806 01:01:12.375412 14330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:33837 every 8 connection(s)
I20260806 01:01:12.376677 14130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:12.385927 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 14130
I20260806 01:01:12.386122 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:01:12.386773 5554 external_mini_cluster.cc:1428] Setting key 4c691fb3c6ecbd68abdbebba15171602
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:12.400410 14334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:12.379348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:50653 (local address 127.5.108.190:46803)
0806 01:01:12.379851 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:12.379856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:12.381063 (+ 1207us) server_negotiation.cc:407] Connection header received
0806 01:01:12.382849 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:12.382855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:12.382943 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:12.383065 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:12.384634 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.385382 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.387109 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.387709 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.387851 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:12.392452 (+ 4601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:12.392460 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:12.392466 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:12.392508 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:12.396462 (+ 3954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:12.397444 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:12.397453 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:12.397458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:12.397627 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:12.398199 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:12.398206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:12.398211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:12.398553 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:12.398731 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:12.399673 (+ 942us) server_negotiation.cc:299] Negotiation successful
0806 01:01:12.399914 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":256,"threads_started":1}
I20260806 01:01:12.401677 14335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:12.380046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46803 (local address 127.5.108.130:50653)
0806 01:01:12.380913 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:12.380950 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:12.382575 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:12.383315 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:12.383323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:12.383887 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:12.384460 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:12.384477 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.385546 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.385552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:12.386971 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:12.386984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.387938 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.387947 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:12.388216 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:12.388933 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:12.388963 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:12.392209 (+ 3246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:12.396613 (+ 4404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:12.396624 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:12.396641 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:12.397295 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:12.397776 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:12.397783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:12.397789 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:12.398050 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:12.398728 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:12.398747 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:12.399194 (+ 447us) client_negotiation.cc:769] Sending connection context
0806 01:01:12.399536 (+ 342us) client_negotiation.cc:240] Negotiation successful
0806 01:01:12.400089 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":391,"threads_started":1}
I20260806 01:01:12.403781 14331 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46803
I20260806 01:01:12.404212 14331 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:12.405222 14331 heartbeater.cc:511] Master 127.5.108.190:46803 requested a full tablet report, sending...
I20260806 01:01:12.407002 13814 ts_manager.cc:194] Registered new tserver with Master: 0ad35a294f5046b7895324e7d686ecb3 (127.5.108.130:33837)
I20260806 01:01:12.408257 13814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:50653
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:01:12.442018 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46803
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:12.765388 14339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:12.765655 14339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:12.765847 14339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:12.772938 14339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:12.773057 14339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:12.773257 14339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:01:12.782887 14339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:12.784032 14339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:12.785288 14339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:12.798180 14349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:12.798216 14345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:12.798182 14346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:12.799506 14339 server_base.cc:1034] running on GCE node
I20260806 01:01:12.800102 14339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:12.801033 14339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:12.802232 14339 hybrid_clock.cc:648] HybridClock initialized: now 1785978072802163 us; error 49 us; skew 500 ppm
I20260806 01:01:12.802546 14339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:12.805598 14339 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:01:12.806892 14339 webserver.cc:460] Webserver started at http://127.5.108.131:38495/ using document root <none> and password file <none>
I20260806 01:01:12.807626 14339 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:12.807798 14339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:12.808099 14339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:12.811084 14339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "1670dfbfb2ec4827ad0cd83a55d84f48"
format_stamp: "Formatted at 2026-08-06 01:01:12 on dist-test-slave-x4qh"
server_key: "41a02e1925e0c0e28570e3d12d1d8e60"
server_key_iv: "503a2bb253d4ada47d09c864016940c9"
server_key_version: "encryptionkey@0"
I20260806 01:01:12.811978 14339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "1670dfbfb2ec4827ad0cd83a55d84f48"
format_stamp: "Formatted at 2026-08-06 01:01:12 on dist-test-slave-x4qh"
server_key: "41a02e1925e0c0e28570e3d12d1d8e60"
server_key_iv: "503a2bb253d4ada47d09c864016940c9"
server_key_version: "encryptionkey@0"
I20260806 01:01:12.816943 14339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:01:12.820246 14357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:12.821467 14339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:01:12.821650 14339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "1670dfbfb2ec4827ad0cd83a55d84f48"
format_stamp: "Formatted at 2026-08-06 01:01:12 on dist-test-slave-x4qh"
server_key: "41a02e1925e0c0e28570e3d12d1d8e60"
server_key_iv: "503a2bb253d4ada47d09c864016940c9"
server_key_version: "encryptionkey@0"
I20260806 01:01:12.821861 14339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:12.838548 14339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:12.842429 14339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:12.842711 14339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:12.843726 14339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:12.845307 14339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:12.845414 14339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:12.845577 14339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:12.845654 14339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:12.881649 14339 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:44809
I20260806 01:01:12.881735 14522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:44809 every 8 connection(s)
I20260806 01:01:12.882911 14339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:12.886545 14121 heartbeater.cc:503] Master 127.5.108.190:46803 was elected leader, sending a full tablet report...
I20260806 01:01:12.891043 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 14339
I20260806 01:01:12.891247 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:01:12.891750 5554 external_mini_cluster.cc:1428] Setting key 6b8a04330fcaeac8af5ac9fb0737a44a
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:12.903136 14334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:12.885528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:58005 (local address 127.5.108.190:46803)
0806 01:01:12.885711 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:12.885716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:12.886891 (+ 1175us) server_negotiation.cc:407] Connection header received
0806 01:01:12.889091 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:12.889096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:12.889180 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:12.889300 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:12.890742 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.891685 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.893557 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.894073 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.894179 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:12.897689 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:12.897694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:12.897697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:12.897733 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:12.900400 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:12.901093 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:12.901099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:12.901105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:12.901248 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:12.901692 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:12.901695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:12.901697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:12.901951 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:12.902085 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:12.902686 (+ 601us) server_negotiation.cc:299] Negotiation successful
0806 01:01:12.902848 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 01:01:12.904022 14529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:12.885919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46803 (local address 127.5.108.131:58005)
0806 01:01:12.886698 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:12.886762 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:12.888808 (+ 2046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:12.889533 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:12.889541 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:12.890020 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:12.890556 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:12.890572 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.891845 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.891850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:12.893364 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:12.893401 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.894328 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.894337 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:12.894662 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:12.895324 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:12.895349 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:12.897517 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:12.900522 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:12.900528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:12.900538 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:12.900919 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:12.901399 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:12.901404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:12.901406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:12.901594 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:12.902068 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:12.902073 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:12.902374 (+ 301us) client_negotiation.cc:769] Sending connection context
0806 01:01:12.902589 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 01:01:12.902921 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":326,"threads_started":1}
I20260806 01:01:12.905440 14525 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46803
I20260806 01:01:12.905764 14525 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:12.906564 14525 heartbeater.cc:511] Master 127.5.108.190:46803 requested a full tablet report, sending...
I20260806 01:01:12.907900 13814 ts_manager.cc:194] Registered new tserver with Master: 1670dfbfb2ec4827ad0cd83a55d84f48 (127.5.108.131:44809)
I20260806 01:01:12.908736 13814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:58005
I20260806 01:01:12.918888 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/test-admin.keytab.
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:12.951790 5554 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 01:01:12.957253 14536 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:12.954561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.5.108.129:35063)
0806 01:01:12.955244 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:12.955251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:12.955275 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:01:12.955336 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:12.955342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:12.955615 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:12.955806 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:12.956774 (+ 968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60214: BlockingRecv error: recv got EOF from 127.0.0.1:60214 (error 108)
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":342,"threads_started":1}
W20260806 01:01:12.957316 5554 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5cc' of type FILE
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:12.958876 5554 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:12.960297 5554 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:01:12.975455 14536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:12.961774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.5.108.129:35063)
0806 01:01:12.961973 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:12.961981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:12.962275 (+ 294us) server_negotiation.cc:407] Connection header received
0806 01:01:12.962451 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:12.962458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:12.962519 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:12.962623 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:12.963667 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.964587 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.965818 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.966408 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.966547 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:12.969882 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:12.969893 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:12.969906 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:12.969935 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:12.972747 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:12.973375 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:12.973382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:12.973384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:12.973503 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:12.973884 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:12.973889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:12.973894 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:12.974227 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:12.974527 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:12.974866 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 01:01:12.975124 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 01:01:12.981817 14536 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:12.979416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60232 (local address 127.5.108.129:35063)
0806 01:01:12.979613 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:12.979619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:12.979826 (+ 207us) server_negotiation.cc:407] Connection header received
0806 01:01:12.979981 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:12.979985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:12.980047 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:12.980132 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:12.981559 (+ 1427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60232: BlockingRecv error: recv got EOF from 127.0.0.1:60232 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 01:01:12.981770 5554 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5cc)
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:12.983646 5554 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:12.984958 5554 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:01:12.998086 14536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:12.986212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.5.108.129:35063)
0806 01:01:12.986398 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:12.986404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:12.986598 (+ 194us) server_negotiation.cc:407] Connection header received
0806 01:01:12.986698 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:12.986701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:12.986757 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:12.986828 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:12.987702 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.988338 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.989539 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:12.989946 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:12.990034 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:12.993409 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:12.993414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:12.993417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:12.993445 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:12.995866 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:12.996451 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:12.996454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:12.996457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:12.996574 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:12.996883 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:12.996886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:12.996887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:12.997112 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:12.997297 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:12.997613 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 01:01:12.997768 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 01:01:13.002207 14536 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:13.000758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60254 (local address 127.5.108.129:35063)
0806 01:01:13.000930 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:13.000936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:13.001057 (+ 121us) server_negotiation.cc:407] Connection header received
0806 01:01:13.001182 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:13.001185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:13.001255 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:13.001339 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:13.002003 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60254: BlockingRecv error: recv got EOF from 127.0.0.1:60254 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 01:01:13.002362 5554 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978022745104-5554-0/krb5kdc/krb5cc' of type FILE
Aug 06 01:01:12 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:13.003868 5554 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 01:01:13 dist-test-slave-x4qh krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:13.005275 5554 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 01:01:13 dist-test-slave-x4qh krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:01:13.019192 14536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:13.006428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.5.108.129:35063)
0806 01:01:13.006632 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:13.006637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:13.006855 (+ 218us) server_negotiation.cc:407] Connection header received
0806 01:01:13.006957 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:13.006960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:13.007007 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:13.007079 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:13.008070 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:13.008873 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:13.010265 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:13.010820 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:13.010918 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:13.014397 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:13.014402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:13.014404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:13.014431 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:13.016965 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:13.017566 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:13.017570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:13.017572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:13.017691 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:13.018038 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:13.018040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:13.018042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:13.018281 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:13.018502 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:13.018679 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 01:01:13.018899 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:01:13.020651 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 13913
I20260806 01:01:13.032410 14536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:13.021336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60272 (local address 127.5.108.129:35063)
0806 01:01:13.021594 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:13.021602 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:13.021885 (+ 283us) server_negotiation.cc:407] Connection header received
0806 01:01:13.022055 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:13.022065 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:13.022111 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:13.022204 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:13.023020 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:13.023657 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:13.024807 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:13.025341 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:13.025471 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:13.027488 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:13.027493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:13.027497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:13.027527 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:13.030132 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:13.030659 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:13.030662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:13.030664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:13.030805 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:13.031140 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:13.031144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:13.031146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:13.031461 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:13.031670 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:13.031869 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 01:01:13.032046 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260806 01:01:13.033051 14109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:13.112715 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 13913
I20260806 01:01:13.134549 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 14130
Aug 06 01:01:13 dist-test-slave-x4qh krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:01:13.149802 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:13.135500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.5.108.130:33837)
0806 01:01:13.136036 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:13.136042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:13.136067 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:01:13.136128 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:13.136134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:13.136350 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:13.136526 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:13.137513 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:13.138621 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:13.139918 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:13.140485 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:13.140631 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:13.144394 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:13.144405 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:13.144418 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:13.144451 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:13.147335 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:13.148008 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:13.148015 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:13.148017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:13.148137 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:13.148506 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:13.148509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:13.148511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:13.148843 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:13.149077 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:13.149301 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 01:01:13.149492 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":264,"threads_started":1}
I20260806 01:01:13.150987 14324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:13.232450 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 14130
I20260806 01:01:13.255975 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 14339
Aug 06 01:01:13 dist-test-slave-x4qh krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:01:13.271018 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:13.257049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.5.108.131:44809)
0806 01:01:13.257793 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:13.257800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:13.257829 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:01:13.257899 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:13.257906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:13.258108 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:13.258256 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:13.259245 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:13.260312 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:13.261439 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:13.261936 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:13.262024 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:13.265786 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:13.265797 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:13.265821 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:13.265851 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:13.268399 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:13.269050 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:13.269057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:13.269059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:13.269183 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:13.269542 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:13.269547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:13.269550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:13.269863 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:13.270174 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:13.270447 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 01:01:13.270684 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":400,"threads_started":1}
I20260806 01:01:13.272265 14516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:13.351554 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 14339
I20260806 01:01:13.376166 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 13766
I20260806 01:01:13.377437 13866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:13.472874 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 13766
2026-08-06T01:01:13Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4074 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:01:13 dist-test-slave-x4qh krb5kdc[14562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:01:13 dist-test-slave-x4qh krb5kdc[14562](info): set up 2 sockets
Aug 06 01:01:13 dist-test-slave-x4qh krb5kdc[14562](info): commencing operation
krb5kdc: starting...
W20260806 01:01:14.720623 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:01:14 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:01:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.190.keytab.
Entry for principal oryx/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.190.keytab.
I20260806 01:01:14.934506 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:46687
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.190.keytab
--principal=oryx/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:46687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:15.259313 14582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:15.259680 14582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:15.259845 14582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:15.266655 14582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:15.266813 14582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:15.266914 14582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:15.267033 14582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:15.267119 14582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:15.276445 14582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:46687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.190.keytab
--principal=oryx/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:46687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:15.277649 14582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:15.278900 14582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:15.291057 14590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:15.291080 14593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:15.291006 14591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:15.291806 14582 server_base.cc:1034] running on GCE node
I20260806 01:01:15.292672 14582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:15.294049 14582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:15.295297 14582 hybrid_clock.cc:648] HybridClock initialized: now 1785978075295242 us; error 38 us; skew 500 ppm
I20260806 01:01:15.295713 14582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:15 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978075, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:15.298764 14582 init.cc:377] Logged in from keytab as oryx/127.5.108.190@KRBTEST.COM (short username oryx)
I20260806 01:01:15.300273 14582 webserver.cc:460] Webserver started at http://127.5.108.190:40951/ using document root <none> and password file <none>
I20260806 01:01:15.300936 14582 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:15.301091 14582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:15.301429 14582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:15.304507 14582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "23def1fd01f54c93ac66ee4d7c22c8c2"
format_stamp: "Formatted at 2026-08-06 01:01:15 on dist-test-slave-x4qh"
server_key: "87d46cc6f08d5519ee958f74df6de47b"
server_key_iv: "8c7035066e99d5245ded4a508cb8b3d3"
server_key_version: "encryptionkey@0"
I20260806 01:01:15.305333 14582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "23def1fd01f54c93ac66ee4d7c22c8c2"
format_stamp: "Formatted at 2026-08-06 01:01:15 on dist-test-slave-x4qh"
server_key: "87d46cc6f08d5519ee958f74df6de47b"
server_key_iv: "8c7035066e99d5245ded4a508cb8b3d3"
server_key_version: "encryptionkey@0"
I20260806 01:01:15.310439 14582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:01:15.313711 14602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:15.314868 14582 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:01:15.315057 14582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "23def1fd01f54c93ac66ee4d7c22c8c2"
format_stamp: "Formatted at 2026-08-06 01:01:15 on dist-test-slave-x4qh"
server_key: "87d46cc6f08d5519ee958f74df6de47b"
server_key_iv: "8c7035066e99d5245ded4a508cb8b3d3"
server_key_version: "encryptionkey@0"
I20260806 01:01:15.315258 14582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:15.328958 14582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:15.332643 14582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:15.332952 14582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:15.352133 14582 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:46687
I20260806 01:01:15.352154 14678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:46687 every 8 connection(s)
I20260806 01:01:15.353472 14582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:15.356882 14679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:15.363948 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 14582
I20260806 01:01:15.364172 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:15.365053 5554 external_mini_cluster.cc:1428] Setting key adfe46ecdaa77f33c4bfa55ef547ce51
I20260806 01:01:15.367023 14679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: Bootstrap starting.
I20260806 01:01:15.369902 14679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:15.371021 14679 log.cc:824] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:15.373845 14679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: No bootstrap required, opened a new log
Aug 06 01:01:15 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:15.382052 14679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } }
I20260806 01:01:15.382534 14679 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:15.382694 14679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23def1fd01f54c93ac66ee4d7c22c8c2, State: Initialized, Role: FOLLOWER
I20260806 01:01:15.383347 14679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } }
I20260806 01:01:15.383657 14679 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:15.383831 14679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:15.384042 14679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:15.386526 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:15.367162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.5.108.190:46687)
0806 01:01:15.368026 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:15.368036 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:15.368076 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:01:15.369738 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:15.369759 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:15.370133 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:15.370441 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:15.372173 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:15.373446 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:15.374718 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:15.375421 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:15.375636 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:15.379432 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:15.379441 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:15.379458 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:15.379501 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:15.382328 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:15.383221 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:15.383238 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:15.383244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:15.383465 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:15.383852 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:15.383857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:15.383861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:15.384483 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:15.384835 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:15.385254 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 01:01:15.385656 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":446,"threads_started":1}
I20260806 01:01:15.386610 14679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } }
I20260806 01:01:15.387159 14679 leader_election.cc:302] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23def1fd01f54c93ac66ee4d7c22c8c2; no voters:
I20260806 01:01:15.387888 14679 leader_election.cc:288] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:15.388036 14686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:15.389257 14686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 1 LEADER]: Becoming Leader. State: Replica: 23def1fd01f54c93ac66ee4d7c22c8c2, State: Running, Role: LEADER
I20260806 01:01:15.389796 14686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } }
I20260806 01:01:15.390307 14679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:15.392879 14688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } }
I20260806 01:01:15.393568 14688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:15.392982 14689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23def1fd01f54c93ac66ee4d7c22c8c2. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } }
I20260806 01:01:15.393729 14689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:15.400822 14694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:15.406201 14694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:15.414634 14694 catalog_manager.cc:1347] Generated new cluster ID: d3daf9e8901a4ca7a4ed8fcaeab76cb8
I20260806 01:01:15.414762 14694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:15.448904 14694 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:15.450111 14694 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:15.461777 14694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: Generated new TSK 0
I20260806 01:01:15.462503 14694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab.
Entry for principal oryx/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab.
I20260806 01:01:15.529201 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46687
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab
--principal=oryx/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:15.864115 14715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:15.864423 14715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:15.864599 14715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:15.871493 14715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:15.871618 14715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:15.871801 14715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:15.881206 14715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab
--principal=oryx/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:15.882396 14715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:15.883816 14715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:15.896739 14721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:15.896806 14726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:15.896854 14724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:15.897696 14715 server_base.cc:1034] running on GCE node
I20260806 01:01:15.898367 14715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:15.899461 14715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:15.900684 14715 hybrid_clock.cc:648] HybridClock initialized: now 1785978075900640 us; error 34 us; skew 500 ppm
I20260806 01:01:15.901050 14715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:15 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978075, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:15.904327 14715 init.cc:377] Logged in from keytab as oryx/127.5.108.129@KRBTEST.COM (short username oryx)
I20260806 01:01:15.906885 14715 webserver.cc:460] Webserver started at http://127.5.108.129:42747/ using document root <none> and password file <none>
I20260806 01:01:15.907994 14715 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:15.908174 14715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:15.908550 14715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:15.913061 14715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "e659777d823d41cc976196dc95dd78c3"
format_stamp: "Formatted at 2026-08-06 01:01:15 on dist-test-slave-x4qh"
server_key: "79274540082961ecb272216849454836"
server_key_iv: "19a19f6b252f330be58fc9f03bbabb96"
server_key_version: "encryptionkey@0"
I20260806 01:01:15.914291 14715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "e659777d823d41cc976196dc95dd78c3"
format_stamp: "Formatted at 2026-08-06 01:01:15 on dist-test-slave-x4qh"
server_key: "79274540082961ecb272216849454836"
server_key_iv: "19a19f6b252f330be58fc9f03bbabb96"
server_key_version: "encryptionkey@0"
I20260806 01:01:15.921144 14715 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260806 01:01:15.925971 14737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:15.927322 14715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 01:01:15.927505 14715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "e659777d823d41cc976196dc95dd78c3"
format_stamp: "Formatted at 2026-08-06 01:01:15 on dist-test-slave-x4qh"
server_key: "79274540082961ecb272216849454836"
server_key_iv: "19a19f6b252f330be58fc9f03bbabb96"
server_key_version: "encryptionkey@0"
I20260806 01:01:15.927798 14715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:15.946161 14715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:15.951992 14715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:15.952310 14715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:15.953469 14715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:15.955421 14715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:15.955529 14715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 01:01:15.955667 14715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:15.955758 14715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:15.987946 14715 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:35491
I20260806 01:01:15.987993 14918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:35491 every 8 connection(s)
I20260806 01:01:15.989215 14715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:15.990020 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 14715
I20260806 01:01:15.990299 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:01:15.991232 5554 external_mini_cluster.cc:1428] Setting key 530d6f6a22034bc698580b42636f621c
Aug 06 01:01:16 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978075, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:16.013094 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:15.992792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:45911 (local address 127.5.108.190:46687)
0806 01:01:15.993288 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:15.993293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:15.994322 (+ 1029us) server_negotiation.cc:407] Connection header received
0806 01:01:15.997280 (+ 2958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:15.997284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:15.997346 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:15.997432 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:15.999433 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.000098 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.002352 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.002718 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.002803 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:16.006915 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:16.006920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:16.006923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:16.006956 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:16.009541 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:16.010660 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:16.010664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:16.010666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:16.010788 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:16.011329 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:16.011332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:16.011334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:16.011657 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:16.011757 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:16.012661 (+ 904us) server_negotiation.cc:299] Negotiation successful
0806 01:01:16.012809 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":330,"threads_started":1}
I20260806 01:01:16.014714 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:15.993355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.129:45911)
0806 01:01:15.994107 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:15.994176 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:15.996952 (+ 2776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:15.997689 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:15.997702 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:15.998451 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:15.999242 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:15.999268 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.000258 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.000266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:16.002219 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:16.002236 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.002918 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.002927 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:16.003233 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:16.004001 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:16.004021 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:16.006684 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:16.009674 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:16.009684 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:16.009699 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:16.010512 (+ 813us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:16.010911 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:16.010919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:16.010925 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:16.011212 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:16.011816 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:16.011828 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:16.012259 (+ 431us) client_negotiation.cc:769] Sending connection context
0806 01:01:16.012561 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 01:01:16.013157 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":284,"threads_started":1}
I20260806 01:01:16.016381 14919 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46687
I20260806 01:01:16.016683 14919 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:16.017309 14919 heartbeater.cc:511] Master 127.5.108.190:46687 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.5.108.130@KRBTEST.COM; defaulting to no policy
I20260806 01:01:16.019435 14629 ts_manager.cc:194] Registered new tserver with Master: e659777d823d41cc976196dc95dd78c3 (127.5.108.129:35491)
I20260806 01:01:16.020864 14629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:45911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab.
Entry for principal oryx/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab.
I20260806 01:01:16.046634 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46687
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab
--principal=oryx/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:16.362217 14931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:16.362484 14931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:16.362613 14931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:16.369313 14931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:16.369426 14931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:16.369594 14931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:16.378537 14931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab
--principal=oryx/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:16.379734 14931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:16.380925 14931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:16.393122 14944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:16.393199 14941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:16.393409 14940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:16.394274 14931 server_base.cc:1034] running on GCE node
I20260806 01:01:16.394804 14931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:16.395741 14931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:16.396955 14931 hybrid_clock.cc:648] HybridClock initialized: now 1785978076396920 us; error 47 us; skew 500 ppm
I20260806 01:01:16.397231 14931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:16 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978076, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:16.410375 14931 init.cc:377] Logged in from keytab as oryx/127.5.108.130@KRBTEST.COM (short username oryx)
I20260806 01:01:16.411787 14931 webserver.cc:460] Webserver started at http://127.5.108.130:38801/ using document root <none> and password file <none>
I20260806 01:01:16.412468 14931 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:16.412608 14931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:16.412904 14931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:16.415803 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "aad0aac9bfef4ac5883682d342760f17"
format_stamp: "Formatted at 2026-08-06 01:01:16 on dist-test-slave-x4qh"
server_key: "f40e15c58f5fa9d30624da1bfb969f8e"
server_key_iv: "498fd6f7e1dccc2ddece1bef462e3232"
server_key_version: "encryptionkey@0"
I20260806 01:01:16.416553 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "aad0aac9bfef4ac5883682d342760f17"
format_stamp: "Formatted at 2026-08-06 01:01:16 on dist-test-slave-x4qh"
server_key: "f40e15c58f5fa9d30624da1bfb969f8e"
server_key_iv: "498fd6f7e1dccc2ddece1bef462e3232"
server_key_version: "encryptionkey@0"
I20260806 01:01:16.421475 14931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:01:16.424656 14953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:16.425894 14931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:16.426060 14931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "aad0aac9bfef4ac5883682d342760f17"
format_stamp: "Formatted at 2026-08-06 01:01:16 on dist-test-slave-x4qh"
server_key: "f40e15c58f5fa9d30624da1bfb969f8e"
server_key_iv: "498fd6f7e1dccc2ddece1bef462e3232"
server_key_version: "encryptionkey@0"
I20260806 01:01:16.426270 14931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:16.448912 14931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:16.452296 14931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:16.452545 14931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:16.453431 14931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:16.454676 14931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:16.454777 14931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:16.454916 14931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:16.454988 14931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:16.487597 14931 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:44927
I20260806 01:01:16.487669 15119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:44927 every 8 connection(s)
I20260806 01:01:16.488750 14931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:16.494925 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 14931
I20260806 01:01:16.495123 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:01:16.495748 5554 external_mini_cluster.cc:1428] Setting key de243fefa57583f92c0ef031d1bcb5a4
Aug 06 01:01:16 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978076, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:16.509866 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:16.491166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:48291 (local address 127.5.108.190:46687)
0806 01:01:16.491396 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:16.491416 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:16.492695 (+ 1279us) server_negotiation.cc:407] Connection header received
0806 01:01:16.494520 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:16.494524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:16.494610 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:16.494713 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:16.496646 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.497446 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.499003 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.499785 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.499925 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:16.503265 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:16.503275 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:16.503283 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:16.503327 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:16.506869 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:16.507648 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:16.507656 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:16.507662 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:16.507832 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:16.508297 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:16.508301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:16.508303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:16.508551 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:16.508741 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:16.509358 (+ 617us) server_negotiation.cc:299] Negotiation successful
0806 01:01:16.509564 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:01:16.510730 15124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:16.491693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.130:48291)
0806 01:01:16.492477 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:16.492524 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:16.494229 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:16.494985 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:16.494997 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:16.495739 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:16.496481 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:16.496501 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.497616 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.497623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:16.498869 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:16.498882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.500010 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.500015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:16.500207 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:16.500889 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:16.500910 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:16.503047 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:16.507015 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:16.507022 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:16.507031 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:16.507527 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:16.507961 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:16.507965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:16.507967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:16.508135 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:16.508722 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:16.508729 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:16.509023 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 01:01:16.509253 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 01:01:16.509711 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":318,"threads_started":1}
I20260806 01:01:16.512115 15121 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46687
I20260806 01:01:16.512399 15121 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:16.513051 15121 heartbeater.cc:511] Master 127.5.108.190:46687 requested a full tablet report, sending...
I20260806 01:01:16.514379 14629 ts_manager.cc:194] Registered new tserver with Master: aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927)
I20260806 01:01:16.515192 14629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:48291
WARNING: no policy specified for oryx/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab.
Entry for principal oryx/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab.
I20260806 01:01:16.548470 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46687
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab
--principal=oryx/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:16.868589 15128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:16.868830 15128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:16.868916 15128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:16.875671 15128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:16.875767 15128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:16.875871 15128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:01:16.884600 15128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab
--principal=oryx/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:16.885802 15128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:16.886984 15128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:16.899709 15134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:16.899680 15140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:16.899644 15135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:16.900511 15128 server_base.cc:1034] running on GCE node
I20260806 01:01:16.901068 15128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:16.901899 15128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:16.903108 15128 hybrid_clock.cc:648] HybridClock initialized: now 1785978076903056 us; error 60 us; skew 500 ppm
I20260806 01:01:16.903447 15128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:16 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978076, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:16.906517 15128 init.cc:377] Logged in from keytab as oryx/127.5.108.131@KRBTEST.COM (short username oryx)
I20260806 01:01:16.907831 15128 webserver.cc:460] Webserver started at http://127.5.108.131:41869/ using document root <none> and password file <none>
I20260806 01:01:16.908470 15128 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:16.908609 15128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:16.908896 15128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:16.911777 15128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "7d6e509992994b87a41b46c457c8d74b"
format_stamp: "Formatted at 2026-08-06 01:01:16 on dist-test-slave-x4qh"
server_key: "540c28c2d9298a29a01ed31f95b40c64"
server_key_iv: "3ca7ceec7676fb9b69cafbbe2183774b"
server_key_version: "encryptionkey@0"
I20260806 01:01:16.912595 15128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "7d6e509992994b87a41b46c457c8d74b"
format_stamp: "Formatted at 2026-08-06 01:01:16 on dist-test-slave-x4qh"
server_key: "540c28c2d9298a29a01ed31f95b40c64"
server_key_iv: "3ca7ceec7676fb9b69cafbbe2183774b"
server_key_version: "encryptionkey@0"
I20260806 01:01:16.917344 15128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:01:16.920655 15147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:16.921847 15128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:16.922025 15128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "7d6e509992994b87a41b46c457c8d74b"
format_stamp: "Formatted at 2026-08-06 01:01:16 on dist-test-slave-x4qh"
server_key: "540c28c2d9298a29a01ed31f95b40c64"
server_key_iv: "3ca7ceec7676fb9b69cafbbe2183774b"
server_key_version: "encryptionkey@0"
I20260806 01:01:16.922361 15128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:16.936779 15128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:16.940887 15128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:16.941177 15128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:16.942092 15128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:16.943315 15128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:16.943468 15128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:16.943605 15128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:16.943717 15128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:16.976593 15128 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:45995
I20260806 01:01:16.976595 15339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:45995 every 8 connection(s)
I20260806 01:01:16.977866 15128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:16.986972 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 15128
I20260806 01:01:16.987195 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:01:16.987799 5554 external_mini_cluster.cc:1428] Setting key 7e2602e8f303a0038a34f935bf9e264e
Aug 06 01:01:16 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978076, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:16.998546 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:16.980378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:38183 (local address 127.5.108.190:46687)
0806 01:01:16.980582 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:16.980587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:16.981587 (+ 1000us) server_negotiation.cc:407] Connection header received
0806 01:01:16.983266 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:16.983270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:16.983350 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:16.983477 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:16.984833 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.985635 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.987637 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.988233 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.988384 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:16.992740 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:16.992746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:16.992750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:16.992786 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:16.995208 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:16.995947 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:16.995951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:16.995954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:16.996091 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:16.996627 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:16.996636 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:16.996638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:16.996994 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:16.997155 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:16.997888 (+ 733us) server_negotiation.cc:299] Negotiation successful
0806 01:01:16.998115 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:01:16.999554 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:16.980702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.131:38183)
0806 01:01:16.981417 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:16.981462 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:16.983018 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:16.983680 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:16.983687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:16.984162 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:16.984693 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:16.984710 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.985820 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.985825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:16.987485 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:16.987502 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:16.988809 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:16.988815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:16.989064 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:16.990049 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:16.990071 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:16.992555 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:16.995346 (+ 2791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:16.995353 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:16.995363 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:16.995828 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:16.996233 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:16.996238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:16.996241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:16.996501 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:16.997134 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:16.997141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:16.997441 (+ 300us) client_negotiation.cc:769] Sending connection context
0806 01:01:16.997733 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 01:01:16.998291 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":277,"threads_started":1}
I20260806 01:01:17.000707 15340 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46687
I20260806 01:01:17.001000 15340 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:17.001869 15340 heartbeater.cc:511] Master 127.5.108.190:46687 requested a full tablet report, sending...
I20260806 01:01:17.003486 14629 ts_manager.cc:194] Registered new tserver with Master: 7d6e509992994b87a41b46c457c8d74b (127.5.108.131:45995)
I20260806 01:01:17.004247 14629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:38183
I20260806 01:01:17.014993 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:01:17.023662 14919 heartbeater.cc:503] Master 127.5.108.190:46687 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.5.108.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.189.keytab.
Entry for principal oryx/127.5.108.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.189.keytab.
I20260806 01:01:17.068503 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.189:43679
--webserver_interface=127.5.108.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.189.keytab
--principal=oryx/127.5.108.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:46687,127.5.108.189:43679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:17.389519 15346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:17.389839 15346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:17.389992 15346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:17.396929 15346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:17.397060 15346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:17.397150 15346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:17.397243 15346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:17.397310 15346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:17.406440 15346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:46687,127.5.108.189:43679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.189.keytab
--principal=oryx/127.5.108.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.189:43679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.108.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
W20260806 01:01:17.406813 15346 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.108.190:46687,127.5.108.189:43679'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 01:01:17.407892 15346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:17.409168 15346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:17.420652 15358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:17.420740 15354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:17.420879 15353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:17.422238 15346 server_base.cc:1034] running on GCE node
I20260806 01:01:17.423010 15346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:17.424365 15346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:17.425601 15346 hybrid_clock.cc:648] HybridClock initialized: now 1785978077425538 us; error 28 us; skew 500 ppm
I20260806 01:01:17.426020 15346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:17 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978077, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:17.428975 15346 init.cc:377] Logged in from keytab as oryx/127.5.108.189@KRBTEST.COM (short username oryx)
I20260806 01:01:17.430390 15346 webserver.cc:460] Webserver started at http://127.5.108.189:33097/ using document root <none> and password file <none>
I20260806 01:01:17.431005 15346 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:17.431139 15346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:17.431497 15346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:17.434285 15346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data/instance:
uuid: "bc5c4f325eaa4df3917d92ebc48d5c41"
format_stamp: "Formatted at 2026-08-06 01:01:17 on dist-test-slave-x4qh"
server_key: "856df227dc61aa139974d8bfb4b6d00a"
server_key_iv: "1883c4119c3ca0dc890f4b53a42fc647"
server_key_version: "encryptionkey@0"
I20260806 01:01:17.435002 15346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal/instance:
uuid: "bc5c4f325eaa4df3917d92ebc48d5c41"
format_stamp: "Formatted at 2026-08-06 01:01:17 on dist-test-slave-x4qh"
server_key: "856df227dc61aa139974d8bfb4b6d00a"
server_key_iv: "1883c4119c3ca0dc890f4b53a42fc647"
server_key_version: "encryptionkey@0"
I20260806 01:01:17.439826 15346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260806 01:01:17.442977 15371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:17.444164 15346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 01:01:17.444352 15346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal
uuid: "bc5c4f325eaa4df3917d92ebc48d5c41"
format_stamp: "Formatted at 2026-08-06 01:01:17 on dist-test-slave-x4qh"
server_key: "856df227dc61aa139974d8bfb4b6d00a"
server_key_iv: "1883c4119c3ca0dc890f4b53a42fc647"
server_key_version: "encryptionkey@0"
I20260806 01:01:17.444552 15346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:17.457944 15346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:17.462218 15346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:17.462472 15346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:17.478868 15346 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.189:43679
I20260806 01:01:17.478885 15445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.189:43679 every 8 connection(s)
I20260806 01:01:17.480263 15346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
I20260806 01:01:17.483639 15447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:17.487900 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 15346
I20260806 01:01:17.490855 15447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:01:17 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978077, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.189@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
Aug 06 01:01:17 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.189@KRBTEST.COM
I20260806 01:01:17.517182 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.494331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39738 (local address 127.5.108.190:46687)
0806 01:01:17.494532 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:17.494538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:17.496903 (+ 2365us) server_negotiation.cc:407] Connection header received
0806 01:01:17.499499 (+ 2596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:17.499503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:17.499568 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:17.499662 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:17.500922 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.501813 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.503877 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.504556 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.504652 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.509517 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:17.509524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:17.509528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:17.509584 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:17.513331 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.514288 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.514294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.514298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.514464 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.515086 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.515091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.515096 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.515599 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:17.515755 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:17.516728 (+ 973us) server_negotiation.cc:299] Negotiation successful
0806 01:01:17.516877 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:01:17.517587 15121 heartbeater.cc:503] Master 127.5.108.190:46687 was elected leader, sending a full tablet report...
I20260806 01:01:17.518949 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.494408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:39738)
0806 01:01:17.496713 (+ 2305us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:17.496770 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:17.499205 (+ 2435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:17.499751 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:17.499758 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:17.500449 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:17.500752 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.500768 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.501958 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.501963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.503750 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.503761 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.504764 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.504773 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.505113 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.506134 (+ 1021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:17.506169 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:17.509280 (+ 3111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:17.513507 (+ 4227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.513517 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.513527 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.514141 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.514601 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.514607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.514612 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.514854 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.515753 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:17.515796 (+ 43us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:17.516351 (+ 555us) client_negotiation.cc:769] Sending connection context
0806 01:01:17.516638 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 01:01:17.517159 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2000,"mutex_wait_us":128,"thread_start_us":1825,"threads_started":1}
I20260806 01:01:17.519419 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.490064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.5.108.189:43679)
0806 01:01:17.497590 (+ 7526us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:17.497602 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:17.497958 (+ 356us) server_negotiation.cc:407] Connection header received
0806 01:01:17.499205 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:17.499233 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:17.499538 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:17.499769 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:17.501389 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.503097 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.504529 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.505424 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.505544 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.512117 (+ 6573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:17.512127 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:17.512141 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:17.512192 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:17.515923 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.516792 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.516798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.516801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.516963 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.517434 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.517440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.517442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.517790 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:17.518107 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:17.518729 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 01:01:17.518986 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":7288,"thread_start_us":7070,"threads_started":1}
I20260806 01:01:17.520632 15447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:01:17 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978077, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.189@KRBTEST.COM for oryx/127.5.108.189@KRBTEST.COM
I20260806 01:01:17.531858 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.521375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.0.0.1:35868)
0806 01:01:17.521555 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:17.521572 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:17.521686 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:17.522016 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:17.522020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:17.522368 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:17.522488 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.522495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.523332 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.523335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.524142 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.524153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.524765 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.524769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.524891 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.525420 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:17.525436 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:17.527327 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:17.529930 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.529951 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.529953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.530313 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.530594 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.530597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.530600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.530778 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.531206 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:17.531209 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:17.531340 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 01:01:17.531486 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 01:01:17.531659 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 01:01:17.531981 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.521290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35868 (local address 127.5.108.189:43679)
0806 01:01:17.521457 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:17.521462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:17.521777 (+ 315us) server_negotiation.cc:407] Connection header received
0806 01:01:17.521815 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:17.521818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:17.521873 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:17.521940 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:17.522609 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.523224 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.524257 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.524650 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.524741 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.527531 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:17.527535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:17.527538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:17.527562 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:17.529794 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.530399 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.530403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.530405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.530508 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.530871 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.530875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.530876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.531101 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:17.531218 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:17.531588 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 01:01:17.531754 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":55}
I20260806 01:01:17.536140 15447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: Bootstrap starting.
I20260806 01:01:17.538882 15447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:17.539974 15447 log.cc:824] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:17.542271 15447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: No bootstrap required, opened a new log
I20260806 01:01:17.549149 15447 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } }
I20260806 01:01:17.549443 15447 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:17.549552 15447 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5c4f325eaa4df3917d92ebc48d5c41, State: Initialized, Role: FOLLOWER
I20260806 01:01:17.550058 15447 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } }
I20260806 01:01:17.550848 15455 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } } }
I20260806 01:01:17.551167 15455 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:01:17.551760 15447 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:01:17.560035 15473 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:01:17.560154 15473 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:01:17.563494 15346 master_runner.cc:428] Detected that this master bc5c4f325eaa4df3917d92ebc48d5c41 is joining an existing cluster
I20260806 01:01:17.563598 15346 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.108.189:43679
I20260806 01:01:17.584801 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.567289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39742 (local address 127.5.108.190:46687)
0806 01:01:17.567508 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:17.567514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:17.568632 (+ 1118us) server_negotiation.cc:407] Connection header received
0806 01:01:17.568887 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:17.568895 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:17.568990 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:17.569088 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:17.570685 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.571840 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.573761 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.574450 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.574542 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.577483 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:17.577489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:17.577493 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:17.577530 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:17.581411 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.582405 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.582411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.582416 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.582560 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.583038 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.583043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.583046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.583427 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:17.583535 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:17.584223 (+ 688us) server_negotiation.cc:299] Negotiation successful
0806 01:01:17.584458 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 01:01:17.584865 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.568337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.5.108.189:43679)
0806 01:01:17.568486 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:17.568493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:17.569356 (+ 863us) server_negotiation.cc:407] Connection header received
0806 01:01:17.569522 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:17.569527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:17.569608 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:17.569730 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:17.570852 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.571596 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.573554 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.574204 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.574303 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.577474 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:17.577480 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:17.577484 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:17.577517 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:17.581411 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.582392 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.582397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.582400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.582545 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.583024 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.583028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.583030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.583418 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:17.583535 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:17.584149 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 01:01:17.584392 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:01:17.585850 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.567498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.0.0.1:35870)
0806 01:01:17.569253 (+ 1755us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:17.569269 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:17.569386 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:17.569770 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:17.569774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:17.570404 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:17.570735 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.570744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.571762 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.571767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.573401 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.573423 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.574616 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.574622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.574836 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.575815 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:17.575886 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:17.577279 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:17.581553 (+ 4274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.581559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.581563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.582257 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.582698 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.582703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.582706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.582909 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.583618 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:17.583623 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:17.583882 (+ 259us) client_negotiation.cc:769] Sending connection context
0806 01:01:17.585066 (+ 1184us) client_negotiation.cc:240] Negotiation successful
0806 01:01:17.585346 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1631,"mutex_wait_us":92,"spinlock_wait_cycles":27392,"thread_start_us":1413,"threads_started":1}
I20260806 01:01:17.586279 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.567466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:39742)
0806 01:01:17.568528 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:17.568546 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:17.568759 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:17.569292 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:17.569298 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:17.570097 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:17.570516 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.570529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.571975 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.571980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.573622 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.573633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.574816 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.574821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.575095 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.575823 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:17.575841 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:17.577279 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:17.581553 (+ 4274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.581559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.581563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.582257 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.582698 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.582703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.582706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.582909 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.583619 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:17.583623 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:17.583882 (+ 259us) client_negotiation.cc:769] Sending connection context
0806 01:01:17.585306 (+ 1424us) client_negotiation.cc:240] Negotiation successful
0806 01:01:17.585580 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":66,"spinlock_wait_cycles":43008,"thread_start_us":1025,"threads_started":1}
WARNING: no policy specified for oryx/127.5.108.188@KRBTEST.COM; defaulting to no policy
Aug 06 01:01:17 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978075, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.190@KRBTEST.COM for oryx/127.5.108.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.188@KRBTEST.COM" created.
I20260806 01:01:17.609068 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.593338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.5.108.189:43679)
0806 01:01:17.593493 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:17.593498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:17.594050 (+ 552us) server_negotiation.cc:407] Connection header received
0806 01:01:17.594394 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:17.594401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:17.594492 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:17.594623 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:17.595955 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.597078 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.599056 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.599957 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.600106 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.603631 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:17.603636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:17.603639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:17.603667 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:17.606540 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.607191 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.607196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.607199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.607339 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.607717 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.607723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.607727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.607973 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:17.608129 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:17.608555 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 01:01:17.608739 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:01:17.609122 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.592928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.0.0.1:35882)
0806 01:01:17.593911 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:17.593960 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:17.594201 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:17.594681 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:17.594688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:17.595498 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:17.595791 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.595806 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.597241 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.597246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.598920 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.598934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.600128 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.600134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.600312 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.600936 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:17.600957 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:17.603461 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:17.606675 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.606687 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.606693 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.607070 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.607476 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.607481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.607484 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.607614 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.608115 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:17.608123 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:17.608291 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 01:01:17.608462 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 01:01:17.608746 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":628,"threads_started":1}
I20260806 01:01:17.610750 14629 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.108.189:43679: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41"
member_type: NON_VOTER
last_known_addr {
host: "127.5.108.189"
port: 43679
}
attrs {
promote: true
}
}
dest_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2"
cas_config_opid_index: -1
I20260806 01:01:17.611900 14629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } }
I20260806 01:01:17.616719 15492 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bc5c4f325eaa4df3917d92ebc48d5c41 (127.5.108.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } } }
I20260806 01:01:17.617777 15409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:17.622974 15409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Refusing update from remote peer 23def1fd01f54c93ac66ee4d7c22c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 01:01:17.626114 15490 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 01:01:17.633358 15492 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.108.189:43679
I20260806 01:01:17.633884 15490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } } }
I20260806 01:01:17.634184 15490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:17.634287 15346 master.cc:537] Master@127.5.108.189:43679 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.188.keytab.
Entry for principal oryx/127.5.108.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.188.keytab.
I20260806 01:01:17.650513 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.188:34699
--webserver_interface=127.5.108.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.188.keytab
--principal=oryx/127.5.108.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:46687,127.5.108.189:43679,127.5.108.188:34699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:01:17.650180 15409 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bc5c4f325eaa4df3917d92ebc48d5c41 (127.5.108.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } } }
I20260806 01:01:17.655242 15502 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } } }
I20260806 01:01:17.656018 15498 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } } }
I20260806 01:01:17.656373 15498 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:01:17.656533 15455 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23def1fd01f54c93ac66ee4d7c22c8c2. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } } }
I20260806 01:01:17.656736 15455 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:01:17.656864 15502 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:01:17.673847 15346 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 LEARNER]: Raft consensus shutting down.
I20260806 01:01:17.674430 15346 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 LEARNER]: Raft consensus is shut down!
I20260806 01:01:17.674618 15346 tablet_replica.cc:330] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: stopping tablet replica
I20260806 01:01:17.683029 15346 master.cc:552] Master@127.5.108.189:43679 shutdown complete.
I20260806 01:01:17.683164 15346 master_runner.cc:305] Clearing existing system tablet
I20260806 01:01:17.683638 15346 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:17.683727 15346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:17.686697 15346 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:01:17.688390 15506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:17.688812 15346 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:17.689239 15346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 01:01:17.689368 15346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal
uuid: "bc5c4f325eaa4df3917d92ebc48d5c41"
format_stamp: "Formatted at 2026-08-06 01:01:17 on dist-test-slave-x4qh"
server_key: "856df227dc61aa139974d8bfb4b6d00a"
server_key_iv: "1883c4119c3ca0dc890f4b53a42fc647"
server_key_version: "encryptionkey@0"
I20260806 01:01:17.694342 15346 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:17.701232 15346 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 01:01:17.701470 15346 log.cc:1196] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 01:01:17.701977 15346 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: Deleting consensus metadata
I20260806 01:01:17.702306 15346 master_runner.cc:315] Copying system tablet from 127.5.108.190:46687
I20260806 01:01:17.705133 15346 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: tablet copy: Beginning tablet copy session from remote peer at address 127.5.108.190:46687
I20260806 01:01:17.722810 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.706251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.5.108.190:46687)
0806 01:01:17.706476 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:17.706483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:17.707973 (+ 1490us) server_negotiation.cc:407] Connection header received
0806 01:01:17.708268 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:17.708276 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:17.708370 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:17.708442 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:17.709731 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.710885 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.712748 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.713326 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.713446 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.715760 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:17.715769 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:17.715777 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:17.715824 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:17.719838 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.720590 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.720596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.720599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.720746 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:17.721245 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:17.721250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:17.721253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:17.721590 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:17.721757 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:17.722244 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 01:01:17.722468 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:01:17.722841 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:17.706182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:39758)
0806 01:01:17.707854 (+ 1672us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:17.707879 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:17.708088 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:17.708670 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:17.708675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:17.709240 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:17.709522 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.709535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.711056 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.711061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.712567 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:17.712587 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:17.713483 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:17.713489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:17.713652 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:17.714358 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:17.714385 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:17.715520 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:17.719960 (+ 4440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.719966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.719970 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.720449 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.720888 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:17.720894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:17.720904 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:17.721094 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:17.721744 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:17.721751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:17.721928 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 01:01:17.722083 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 01:01:17.722416 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1517,"thread_start_us":1383,"threads_started":1}
I20260806 01:01:17.724095 14657 tablet_copy_service.cc:140] P 23def1fd01f54c93ac66ee4d7c22c8c2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bc5c4f325eaa4df3917d92ebc48d5c41 ({username='oryx', principal='oryx/127.5.108.189@KRBTEST.COM'} at 127.0.0.1:39758)
I20260806 01:01:17.724534 14657 tablet_copy_service.cc:161] P 23def1fd01f54c93ac66ee4d7c22c8c2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bc5c4f325eaa4df3917d92ebc48d5c41 at {username='oryx', principal='oryx/127.5.108.189@KRBTEST.COM'} at 127.0.0.1:39758: session id = bc5c4f325eaa4df3917d92ebc48d5c41-00000000000000000000000000000000
I20260806 01:01:17.729382 14657 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 01:01:17.731946 15346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 01:01:17.738792 15346 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: tablet copy: Starting download of 0 data blocks...
I20260806 01:01:17.739205 15346 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: tablet copy: Starting download of 1 WAL segments...
I20260806 01:01:17.742765 15346 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 01:01:17.747299 14657 tablet_copy_service.cc:342] P 23def1fd01f54c93ac66ee4d7c22c8c2: Request end of tablet copy session bc5c4f325eaa4df3917d92ebc48d5c41-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.108.189@KRBTEST.COM'} at 127.0.0.1:39758
I20260806 01:01:17.747594 14657 tablet_copy_service.cc:433] P 23def1fd01f54c93ac66ee4d7c22c8c2: ending tablet copy session bc5c4f325eaa4df3917d92ebc48d5c41-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bc5c4f325eaa4df3917d92ebc48d5c41
I20260806 01:01:17.759747 15346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:17.762274 15515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:17.763533 15517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:17.770694 15519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:17.770992 15346 server_base.cc:1034] running on GCE node
I20260806 01:01:17.771531 15346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:17.772133 15346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:17.773380 15346 hybrid_clock.cc:648] HybridClock initialized: now 1785978077773293 us; error 49 us; skew 500 ppm
I20260806 01:01:17.773620 15346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:17 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978077, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:17.775928 15346 init.cc:377] Logged in from keytab as oryx/127.5.108.189@KRBTEST.COM (short username oryx)
I20260806 01:01:17.779464 15346 webserver.cc:460] Webserver started at http://127.5.108.189:43519/ using document root <none> and password file <none>
I20260806 01:01:17.779877 15346 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:17.779964 15346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:17.782877 15346 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:01:17.785168 15527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:17.785655 15346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 01:01:17.785835 15346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal
uuid: "bc5c4f325eaa4df3917d92ebc48d5c41"
format_stamp: "Formatted at 2026-08-06 01:01:17 on dist-test-slave-x4qh"
server_key: "856df227dc61aa139974d8bfb4b6d00a"
server_key_iv: "1883c4119c3ca0dc890f4b53a42fc647"
server_key_version: "encryptionkey@0"
I20260806 01:01:17.786070 15346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:17.799149 15346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:17.803352 15346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:17.825736 15346 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.189:43679
I20260806 01:01:17.825830 15612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.189:43679 every 8 connection(s)
I20260806 01:01:17.826857 15346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
I20260806 01:01:17.832587 15615 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:01:17.836930 15615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: Bootstrap starting.
I20260806 01:01:17.851769 15615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:01:17.852321 15615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: Bootstrap complete.
I20260806 01:01:17.853404 15615 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } }
I20260806 01:01:17.853677 15615 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bc5c4f325eaa4df3917d92ebc48d5c41, State: Initialized, Role: LEARNER
I20260806 01:01:17.854005 15615 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } }
I20260806 01:01:17.854871 15618 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } } }
I20260806 01:01:17.855248 15615 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:17.855294 15618 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:01:17.863894 15636 catalog_manager.cc:1259] Loaded cluster ID: d3daf9e8901a4ca7a4ed8fcaeab76cb8
I20260806 01:01:17.863991 15636 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: loading cluster ID for follower catalog manager: success
I20260806 01:01:17.868105 15636 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: acquiring CA information for follower catalog manager: success
I20260806 01:01:17.872046 15636 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:01:18.006619 15340 heartbeater.cc:503] Master 127.5.108.190:46687 was elected leader, sending a full tablet report...
W20260806 01:01:18.106951 15503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:18.107250 15503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:18.107443 15503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:18.114691 15503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:18.114792 15503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:18.114856 15503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:18.114915 15503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:18.114979 15503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:18.125553 15503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:46687,127.5.108.189:43679,127.5.108.188:34699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.188.keytab
--principal=oryx/127.5.108.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.188:34699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.108.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:18.127043 15503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:18.128479 15503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:18.142657 15642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:18.142601 15645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:18.142820 15647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:18.144297 15503 server_base.cc:1034] running on GCE node
I20260806 01:01:18.145224 15503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:18.146580 15503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:18.147842 15503 hybrid_clock.cc:648] HybridClock initialized: now 1785978078147775 us; error 36 us; skew 500 ppm
I20260806 01:01:18.148224 15503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:18 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978078, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:18.152020 15503 init.cc:377] Logged in from keytab as oryx/127.5.108.188@KRBTEST.COM (short username oryx)
I20260806 01:01:18.153647 15503 webserver.cc:460] Webserver started at http://127.5.108.188:39279/ using document root <none> and password file <none>
I20260806 01:01:18.154311 15503 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:18.154477 15503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:18.154764 15503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:18.158548 15503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data/instance:
uuid: "a9a9b8f63bc94143a5271806507d1c15"
format_stamp: "Formatted at 2026-08-06 01:01:18 on dist-test-slave-x4qh"
server_key: "34921e34177b46b023d79b19f363726a"
server_key_iv: "7fd8e4439720f8c67d97791f1b0cea1a"
server_key_version: "encryptionkey@0"
I20260806 01:01:18.159786 15503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal/instance:
uuid: "a9a9b8f63bc94143a5271806507d1c15"
format_stamp: "Formatted at 2026-08-06 01:01:18 on dist-test-slave-x4qh"
server_key: "34921e34177b46b023d79b19f363726a"
server_key_iv: "7fd8e4439720f8c67d97791f1b0cea1a"
server_key_version: "encryptionkey@0"
I20260806 01:01:18.165083 15503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 01:01:18.168383 15659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:18.169533 15503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260806 01:01:18.169667 15503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal
uuid: "a9a9b8f63bc94143a5271806507d1c15"
format_stamp: "Formatted at 2026-08-06 01:01:18 on dist-test-slave-x4qh"
server_key: "34921e34177b46b023d79b19f363726a"
server_key_iv: "7fd8e4439720f8c67d97791f1b0cea1a"
server_key_version: "encryptionkey@0"
I20260806 01:01:18.169808 15503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:18.189563 15503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:18.194113 15503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:18.194396 15503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:18.214629 15714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } } }
I20260806 01:01:18.214978 15714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:18.228801 15503 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.188:34699
I20260806 01:01:18.228969 15749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.188:34699 every 8 connection(s)
I20260806 01:01:18.230455 15503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
I20260806 01:01:18.233325 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.214927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35890 (local address 127.5.108.189:43679)
0806 01:01:18.223744 (+ 8817us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.223754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.224514 (+ 760us) server_negotiation.cc:407] Connection header received
0806 01:01:18.224613 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.224621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.224759 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.224902 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:18.225833 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.227304 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.229960 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.231523 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.231759 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.232072 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.232717 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.232821 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8606,"thread_start_us":8456,"threads_started":1}
I20260806 01:01:18.233325 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.214784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.0.0.1:35890)
0806 01:01:18.224155 (+ 9371us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.224179 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.224383 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.225231 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.225239 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.225281 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:18.225580 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.225597 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.227502 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.227509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.229773 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.229787 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.232214 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.232221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.232461 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.232534 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.232692 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.232873 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9214,"thread_start_us":9123,"threads_started":1}
I20260806 01:01:18.234750 15618 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23def1fd01f54c93ac66ee4d7c22c8c2. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: NON_VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: true } } }
I20260806 01:01:18.235128 15618 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:01:18.235558 15753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:18.235636 15718 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: attempting to promote NON_VOTER bc5c4f325eaa4df3917d92ebc48d5c41 to VOTER
I20260806 01:01:18.236518 15718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } }
I20260806 01:01:18.238597 15567 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 LEARNER]: Refusing update from remote peer 23def1fd01f54c93ac66ee4d7c22c8c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 01:01:18.238791 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 15503
I20260806 01:01:18.239557 15715 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 01:01:18.242820 15718 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bc5c4f325eaa4df3917d92ebc48d5c41 (127.5.108.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } }
I20260806 01:01:18.243510 15567 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bc5c4f325eaa4df3917d92ebc48d5c41 (127.5.108.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } }
I20260806 01:01:18.245718 15753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:01:18.246737 15714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } }
I20260806 01:01:18.246960 15714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:18.247288 15618 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } }
I20260806 01:01:18.247402 15714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } }
I20260806 01:01:18.247550 15618 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:01:18.247586 15714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
Aug 06 01:01:18 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978078, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.188@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
Aug 06 01:01:18 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.188@KRBTEST.COM
I20260806 01:01:18.270768 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.249038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39760 (local address 127.5.108.190:46687)
0806 01:01:18.251798 (+ 2760us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.251805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.251827 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:01:18.252103 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.252111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.252189 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.252317 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.254109 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.255528 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.257684 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.258425 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.258609 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.263182 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.263187 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.263190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.263219 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.267037 (+ 3818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.268078 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.268083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.268087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.268263 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.268769 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.268772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.268774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.269112 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.269273 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.270148 (+ 875us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.270382 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2625,"thread_start_us":2506,"threads_started":1}
I20260806 01:01:18.272076 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.249042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:39760)
0806 01:01:18.250018 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.250067 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.251851 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.252412 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.252419 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.253301 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:18.253874 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.253912 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.255683 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.255687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.257535 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.257552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.258668 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.258674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.258984 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.259952 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:18.259983 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:18.262927 (+ 2944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:18.267223 (+ 4296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.267234 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.267241 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.267934 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.268381 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.268386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.268390 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.268644 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.269268 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:18.269274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:18.269710 (+ 436us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.270014 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.270487 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":31,"thread_start_us":594,"threads_started":1}
I20260806 01:01:18.272122 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.241498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.5.108.188:34699)
0806 01:01:18.249106 (+ 7608us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.249131 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.249932 (+ 801us) server_negotiation.cc:407] Connection header received
0806 01:01:18.251850 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.251880 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.252164 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.252360 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.254304 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.256054 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.257929 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.258904 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.259032 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.264586 (+ 5554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.264598 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.264611 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.264649 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.268233 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.269079 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.269087 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.269090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.269266 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.269750 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.269753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.269755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.270167 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.270533 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.271079 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.271245 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":7334,"thread_start_us":7059,"threads_started":1}
I20260806 01:01:18.273737 15753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:01:18 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978078, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.188@KRBTEST.COM for oryx/127.5.108.189@KRBTEST.COM
I20260806 01:01:18.288482 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.274638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35892 (local address 127.5.108.189:43679)
0806 01:01:18.274862 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.274866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.275437 (+ 571us) server_negotiation.cc:407] Connection header received
0806 01:01:18.275596 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.275600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.275660 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.275735 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.276541 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.277373 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.278748 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.279126 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.279210 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.282464 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.282468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.282471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.282507 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.285686 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.286428 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.286432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.286433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.286639 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.287069 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.287073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.287075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.287353 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.287571 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.287931 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.288099 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 01:01:18.288609 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.275084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.0.0.1:35892)
0806 01:01:18.275320 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.275340 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.275485 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.275775 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.275778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.276219 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:18.276414 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.276424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.277516 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.277519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.278627 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.278639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.279245 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.279249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.279482 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.280057 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:18.280074 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:18.282273 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:18.285804 (+ 3531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.285808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.285811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.286275 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.286750 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.286754 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.286756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.286943 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.287545 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:18.287548 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:18.287716 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.287841 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.288107 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 01:01:18.289673 15753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:01:18 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978078, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.188@KRBTEST.COM for oryx/127.5.108.188@KRBTEST.COM
I20260806 01:01:18.304111 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.290357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.5.108.188:34699)
0806 01:01:18.290492 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.290496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.290779 (+ 283us) server_negotiation.cc:407] Connection header received
0806 01:01:18.290871 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.290874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.290923 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.291036 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.292075 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.292846 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.294107 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.294680 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.294792 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.298642 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.298646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.298649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.298678 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.301562 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.302262 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.302266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.302268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.302402 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.302753 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.302756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.302758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.303046 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.303152 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.303595 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.303773 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 01:01:18.304141 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.290579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:34699 (local address 127.0.0.1:41894)
0806 01:01:18.290693 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.290707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.290792 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.291071 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.291074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.291759 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:18.291929 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.291938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.292960 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.292963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.293942 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.293955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.294830 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.294834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.295056 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.295833 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:18.295851 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:18.298447 (+ 2596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:18.301695 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.301698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.301700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.302166 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.302492 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.302495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.302497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.302640 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.303142 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:18.303145 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:18.303257 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.303472 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.303643 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 01:01:18.309149 15753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Bootstrap starting.
I20260806 01:01:18.312177 15753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:18.313125 15753 log.cc:824] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:18.315829 15753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: No bootstrap required, opened a new log
I20260806 01:01:18.323832 15753 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } }
I20260806 01:01:18.324126 15753 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:18.324216 15753 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a9b8f63bc94143a5271806507d1c15, State: Initialized, Role: FOLLOWER
I20260806 01:01:18.324895 15753 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } }
I20260806 01:01:18.326001 15769 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } } }
I20260806 01:01:18.326475 15769 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:01:18.327616 15753 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:01:18.340349 15786 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:01:18.340564 15786 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:01:18.344053 15503 master_runner.cc:428] Detected that this master a9a9b8f63bc94143a5271806507d1c15 is joining an existing cluster
I20260806 01:01:18.344237 15503 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.108.188:34699
I20260806 01:01:18.381021 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 14715
I20260806 01:01:18.394259 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.349924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.5.108.189:43679)
0806 01:01:18.350121 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.350127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.373139 (+ 23012us) server_negotiation.cc:407] Connection header received
0806 01:01:18.373491 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.373496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.373602 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.373700 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.376427 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.377716 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.380920 (+ 3204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.381848 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.381951 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.385622 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.385628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.385633 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.385706 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.390187 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.391193 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.391198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.391216 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.391483 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.392087 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.392092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.392097 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.392554 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.392705 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.393523 (+ 818us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.393807 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:01:18.395614 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.350219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.0.0.1:35908)
0806 01:01:18.373036 (+ 22817us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.373055 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.373293 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.373986 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.373992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.375498 (+ 1506us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:18.376196 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.376213 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.378217 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.378222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.380720 (+ 2498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.380740 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.382368 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.382377 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.382639 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.383536 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:18.383567 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:18.385355 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:18.390373 (+ 5018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.390378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.390382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.391057 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.391681 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.391686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.391689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.391925 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.392792 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:18.392799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:18.393094 (+ 295us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.394515 (+ 1421us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.394869 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"mutex_wait_us":35,"thread_start_us":1060,"threads_started":1}
I20260806 01:01:18.397274 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.350279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:34699 (local address 127.0.0.1:41902)
0806 01:01:18.373503 (+ 23224us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.373521 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.373658 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.374106 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.374114 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.375462 (+ 1348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:18.376159 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.376175 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.377765 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.377770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.379783 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.379796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.381726 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.381737 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.381975 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.382951 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:18.383029 (+ 78us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:18.385044 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:18.393144 (+ 8100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.393152 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.393158 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.393944 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.394801 (+ 857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.394808 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.394818 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.395002 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.396212 (+ 1210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:18.396217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:18.396373 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.396506 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.396741 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23093,"mutex_wait_us":35,"spinlock_wait_cycles":89728,"thread_start_us":38681,"threads_started":1}
I20260806 01:01:18.398388 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.349611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.5.108.188:34699)
0806 01:01:18.349953 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.349963 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.373601 (+ 23638us) server_negotiation.cc:407] Connection header received
0806 01:01:18.373782 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.373794 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.373861 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.373948 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.376308 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.377561 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.379952 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.380992 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.381111 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.385270 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.385279 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.385291 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.385336 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.392965 (+ 7629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.394094 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.394102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.394112 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.394294 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.395153 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.395161 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.395166 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.395637 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.396156 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.397686 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.397900 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":87}
I20260806 01:01:18.401765 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.349527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39768 (local address 127.5.108.190:46687)
0806 01:01:18.349714 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.349722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.384423 (+ 34701us) server_negotiation.cc:407] Connection header received
0806 01:01:18.384698 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.384706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.384842 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.384926 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.386778 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.388278 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.390893 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.391757 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.391920 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.394712 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.394718 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.394722 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.394764 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.398344 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.399256 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.399262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.399267 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.399455 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.399968 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.399973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.399977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.400363 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.400514 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.401158 (+ 644us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.401367 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:01:18.401854 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.350651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:39768)
0806 01:01:18.384250 (+ 33599us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.384282 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.384523 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.385172 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.385177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.386068 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:18.386551 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.386590 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.388575 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.388584 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.390676 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.390692 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.392282 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.392290 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.392474 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.393265 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:18.393296 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:18.394482 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:18.398483 (+ 4001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.398488 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.398491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.399060 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.399623 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.399630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.399635 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.399824 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.400591 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:18.400595 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:18.400808 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.401011 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.401304 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33415,"thread_start_us":22511,"threads_started":1}
Aug 06 01:01:18 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.129@KRBTEST.COM
Aug 06 01:01:18 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978075, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.190@KRBTEST.COM for oryx/127.5.108.188@KRBTEST.COM
I20260806 01:01:18.417316 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.383156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44288 (local address 127.5.108.129:35491)
0806 01:01:18.396307 (+ 13151us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.396316 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.396345 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:01:18.396415 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.396424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.396731 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.399174 (+ 2443us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.399591 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.401354 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.403053 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.403965 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.404103 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.409286 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.409298 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.409311 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.409363 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.412685 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.413673 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.413683 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.413687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.413993 (+ 306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.414499 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.414517 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.414527 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.415345 (+ 818us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.415727 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.416260 (+ 533us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.416660 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12947,"thread_start_us":13609,"threads_started":1}
I20260806 01:01:18.418939 14913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:18.428423 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.407428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:34699 (local address 127.0.0.1:41914)
0806 01:01:18.407709 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.407727 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.407855 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.408308 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.408314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.408790 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:18.408996 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.409010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.410626 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.410632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.412495 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.412511 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.413793 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.413800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.414129 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.414743 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:18.414760 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:18.418984 (+ 4224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:18.425419 (+ 6435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.425425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.425429 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.426044 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.426500 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.426504 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.426508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.426679 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.427324 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:18.427329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:18.427500 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.427623 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.427882 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260806 01:01:18.428535 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.407590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41914 (local address 127.5.108.188:34699)
0806 01:01:18.407735 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.407741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.407783 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:01:18.408033 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.408038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.408166 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.408271 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.409183 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.410438 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.412651 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.413618 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.413787 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.419253 (+ 5466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.419262 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.419269 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.419316 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.425276 (+ 5960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.426162 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.426167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.426171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.426344 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.426784 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.426788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.426791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.427142 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.427321 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.427751 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.427957 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:01:18.430274 14629 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.108.188:34699: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15"
member_type: NON_VOTER
last_known_addr {
host: "127.5.108.188"
port: 34699
}
attrs {
promote: true
}
}
dest_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2"
cas_config_opid_index: 6
I20260806 01:01:18.431104 14629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } }
I20260806 01:01:18.433351 15567 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Refusing update from remote peer 23def1fd01f54c93ac66ee4d7c22c8c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 01:01:18.433429 15699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:18.434089 15714 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 01:01:18.437260 15715 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER a9a9b8f63bc94143a5271806507d1c15 (127.5.108.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } } }
I20260806 01:01:18.437836 15567 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER a9a9b8f63bc94143a5271806507d1c15 (127.5.108.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } } }
I20260806 01:01:18.440470 15715 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.108.188:34699
I20260806 01:01:18.441021 15503 master.cc:537] Master@127.5.108.188:34699 shutting down...
I20260806 01:01:18.442363 15618 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } } }
I20260806 01:01:18.442642 15618 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:01:18.442540 15715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } } }
I20260806 01:01:18.442857 15715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:18.443143 15718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } } }
I20260806 01:01:18.443746 15718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
W20260806 01:01:18.446640 15668 tablet.cc:2369] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:01:18.447495 15699 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Refusing update from remote peer 23def1fd01f54c93ac66ee4d7c22c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 01:01:18.447917 15769 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:18.448134 15769 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } }
I20260806 01:01:18.448268 15718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:01:18.464038 14609 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 01:01:18.465391 15769 leader_election.cc:288] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23def1fd01f54c93ac66ee4d7c22c8c2 (127.5.108.190:46687), bc5c4f325eaa4df3917d92ebc48d5c41 (127.5.108.189:43679)
I20260806 01:01:18.465925 15567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9a9b8f63bc94143a5271806507d1c15" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" is_pre_election: true
I20260806 01:01:18.466106 14645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9a9b8f63bc94143a5271806507d1c15" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" is_pre_election: true
I20260806 01:01:18.466861 15660 leader_election.cc:302] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a9b8f63bc94143a5271806507d1c15; no voters: 23def1fd01f54c93ac66ee4d7c22c8c2, bc5c4f325eaa4df3917d92ebc48d5c41
I20260806 01:01:18.467528 15769 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 01:01:18.472882 15503 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:18.473253 15503 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:18.473508 15503 tablet_replica.cc:330] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: stopping tablet replica
I20260806 01:01:18.480260 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.479783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:34699 (local address 127.0.0.1:41918)
0806 01:01:18.479967 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.480085 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:34699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 01:01:18.480474 14609 consensus_peers.cc:487] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 -> Peer a9a9b8f63bc94143a5271806507d1c15 (127.5.108.188:34699): Couldn't send request to peer a9a9b8f63bc94143a5271806507d1c15. Status: Network error: Client connection negotiation failed: client connection to 127.5.108.188:34699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 01:01:18.490244 15503 master.cc:552] Master@127.5.108.188:34699 shutdown complete.
I20260806 01:01:18.490319 15503 master_runner.cc:305] Clearing existing system tablet
I20260806 01:01:18.490682 15503 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:18.490783 15503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:18.493634 15503 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:01:18.495270 15809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:18.495690 15503 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:18.495950 15503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:01:18.496053 15503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal
uuid: "a9a9b8f63bc94143a5271806507d1c15"
format_stamp: "Formatted at 2026-08-06 01:01:18 on dist-test-slave-x4qh"
server_key: "34921e34177b46b023d79b19f363726a"
server_key_iv: "7fd8e4439720f8c67d97791f1b0cea1a"
server_key_version: "encryptionkey@0"
I20260806 01:01:18.500371 15503 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:18.507421 15503 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 01:01:18.507678 15503 log.cc:1196] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 01:01:18.508306 15503 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Deleting consensus metadata
I20260806 01:01:18.508605 15503 master_runner.cc:315] Copying system tablet from 127.5.108.190:46687
I20260806 01:01:18.512796 15503 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: tablet copy: Beginning tablet copy session from remote peer at address 127.5.108.190:46687
I20260806 01:01:18.532251 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.513574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39782 (local address 127.5.108.190:46687)
0806 01:01:18.513763 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.513768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.516264 (+ 2496us) server_negotiation.cc:407] Connection header received
0806 01:01:18.516493 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.516499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.516590 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.516692 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:18.518017 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.519300 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.521184 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.521808 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.521890 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.524370 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:18.524394 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:18.524399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:18.524438 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:18.528436 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.529334 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.529339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.529342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.529513 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:18.530009 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:18.530015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:18.530021 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:18.530477 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:18.530729 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.531658 (+ 929us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.531864 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 01:01:18.532845 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.513605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:39782)
0806 01:01:18.516136 (+ 2531us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.516157 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.516334 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.516750 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.516755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.517391 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:18.517858 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.517871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.519503 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.519511 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.521032 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.521045 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.522159 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.522164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.522320 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.523070 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:18.523091 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:18.524164 (+ 1073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:18.528579 (+ 4415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.528584 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.528588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.529163 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.529658 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:18.529663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:18.529667 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:18.529853 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:18.530682 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:18.530687 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:18.530876 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.531030 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.531386 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2330,"thread_start_us":2151,"threads_started":1}
I20260806 01:01:18.532991 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 14715
I20260806 01:01:18.533309 14657 tablet_copy_service.cc:140] P 23def1fd01f54c93ac66ee4d7c22c8c2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a9a9b8f63bc94143a5271806507d1c15 ({username='oryx', principal='oryx/127.5.108.188@KRBTEST.COM'} at 127.0.0.1:39782)
I20260806 01:01:18.533461 14657 tablet_copy_service.cc:161] P 23def1fd01f54c93ac66ee4d7c22c8c2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a9a9b8f63bc94143a5271806507d1c15 at {username='oryx', principal='oryx/127.5.108.188@KRBTEST.COM'} at 127.0.0.1:39782: session id = a9a9b8f63bc94143a5271806507d1c15-00000000000000000000000000000000
I20260806 01:01:18.536921 14657 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 01:01:18.538553 15503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 01:01:18.545392 15503 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: tablet copy: Starting download of 0 data blocks...
I20260806 01:01:18.545670 15503 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: tablet copy: Starting download of 1 WAL segments...
I20260806 01:01:18.548008 15503 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 01:01:18.551992 14657 tablet_copy_service.cc:342] P 23def1fd01f54c93ac66ee4d7c22c8c2: Request end of tablet copy session a9a9b8f63bc94143a5271806507d1c15-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.108.188@KRBTEST.COM'} at 127.0.0.1:39782
I20260806 01:01:18.552124 14657 tablet_copy_service.cc:433] P 23def1fd01f54c93ac66ee4d7c22c8c2: ending tablet copy session a9a9b8f63bc94143a5271806507d1c15-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a9a9b8f63bc94143a5271806507d1c15
I20260806 01:01:18.560572 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:35491
--local_ip_for_outbound_sockets=127.5.108.129
--tserver_master_addrs=127.5.108.190:46687
--webserver_port=42747
--webserver_interface=127.5.108.129
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab
--principal=oryx/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.108.190:46687,127.5.108.189:43679
--tserver_master_addrs=127.5.108.190:46687,127.5.108.189:43679,127.5.108.188:34699 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:01:18.564193 15503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:18.566809 15817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:18.570847 15818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:18.589157 15826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:18.591730 15503 server_base.cc:1034] running on GCE node
I20260806 01:01:18.592240 15503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:18.593055 15503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:18.594322 15503 hybrid_clock.cc:648] HybridClock initialized: now 1785978078594267 us; error 41 us; skew 500 ppm
I20260806 01:01:18.594552 15503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:18 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978078, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:18.596803 15503 init.cc:377] Logged in from keytab as oryx/127.5.108.188@KRBTEST.COM (short username oryx)
I20260806 01:01:18.604278 15503 webserver.cc:460] Webserver started at http://127.5.108.188:45409/ using document root <none> and password file <none>
I20260806 01:01:18.604657 15503 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:18.604771 15503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:18.607887 15503 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:01:18.609727 15836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:18.610145 15503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 01:01:18.610297 15503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal
uuid: "a9a9b8f63bc94143a5271806507d1c15"
format_stamp: "Formatted at 2026-08-06 01:01:18 on dist-test-slave-x4qh"
server_key: "34921e34177b46b023d79b19f363726a"
server_key_iv: "7fd8e4439720f8c67d97791f1b0cea1a"
server_key_version: "encryptionkey@0"
I20260806 01:01:18.610529 15503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:18.625276 15503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:18.629352 15503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:18.667385 15503 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.188:34699
I20260806 01:01:18.667443 15914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.188:34699 every 8 connection(s)
I20260806 01:01:18.668664 15503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
I20260806 01:01:18.674001 15915 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:01:18.677759 15915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Bootstrap starting.
I20260806 01:01:18.696348 15915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:01:18.696882 15915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Bootstrap complete.
I20260806 01:01:18.697950 15915 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } }
I20260806 01:01:18.698269 15915 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a9a9b8f63bc94143a5271806507d1c15, State: Initialized, Role: LEARNER
I20260806 01:01:18.698624 15915 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } }
I20260806 01:01:18.699492 15922 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } } }
I20260806 01:01:18.699922 15915 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:18.699983 15922 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:01:18.712653 15937 catalog_manager.cc:1259] Loaded cluster ID: d3daf9e8901a4ca7a4ed8fcaeab76cb8
I20260806 01:01:18.712764 15937 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: loading cluster ID for follower catalog manager: success
I20260806 01:01:18.716328 15937 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: acquiring CA information for follower catalog manager: success
I20260806 01:01:18.718735 15937 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:01:18.928478 15940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.920595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41934 (local address 127.5.108.188:34699)
0806 01:01:18.921403 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:18.921408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:18.921424 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:01:18.921482 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:18.921486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:18.921550 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:18.921646 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:18.922269 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.923409 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.926476 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.927474 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.927582 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.927766 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:18.928115 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 01:01:18.928215 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":525,"threads_started":1}
I20260806 01:01:18.928597 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:18.920412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:34699 (local address 127.0.0.1:41934)
0806 01:01:18.920636 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:18.920657 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:18.920851 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:18.921755 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:18.921761 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:18.921811 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:18.922097 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.922110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.923570 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.923575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.926254 (+ 2679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:18.926284 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:18.927664 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:18.927670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:18.927919 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:18.927967 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 01:01:18.928096 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 01:01:18.928263 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 01:01:18.929909 15882 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260806 01:01:18.930661 15922 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23def1fd01f54c93ac66ee4d7c22c8c2. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } } }
I20260806 01:01:18.930955 15922 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: This master's current role is: LEARNER
W20260806 01:01:18.970898 15819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:18.971199 15819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:18.971318 15819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:18.978902 15819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:18.979031 15819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:18.979177 15819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:18.989871 15819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab
--principal=oryx/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:35491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=42747
--tserver_master_addrs=127.5.108.190:46687,127.5.108.189:43679,127.5.108.188:34699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:18.991204 15819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:18.992691 15819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:19.011160 15948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:19.011838 15947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:19.012328 15951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:19.012599 15819 server_base.cc:1034] running on GCE node
I20260806 01:01:19.013197 15819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:19.014237 15819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:19.015527 15819 hybrid_clock.cc:648] HybridClock initialized: now 1785978079015420 us; error 43 us; skew 500 ppm
I20260806 01:01:19.015906 15819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978079, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:19.020236 15819 init.cc:377] Logged in from keytab as oryx/127.5.108.129@KRBTEST.COM (short username oryx)
I20260806 01:01:19.022152 15819 webserver.cc:460] Webserver started at http://127.5.108.129:42747/ using document root <none> and password file <none>
I20260806 01:01:19.022938 15819 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:19.023171 15819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:19.029933 15819 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 01:01:19.033200 15965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:19.034518 15819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 01:01:19.034693 15819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "e659777d823d41cc976196dc95dd78c3"
format_stamp: "Formatted at 2026-08-06 01:01:15 on dist-test-slave-x4qh"
server_key: "79274540082961ecb272216849454836"
server_key_iv: "19a19f6b252f330be58fc9f03bbabb96"
server_key_version: "encryptionkey@0"
I20260806 01:01:19.036103 15819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:19.054469 15819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:19.058902 15819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:19.059183 15819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:19.060385 15819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:19.061833 15819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:19.061961 15819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:19.062112 15819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:19.062203 15819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:19.118507 15819 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:35491
I20260806 01:01:19.118602 16155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:35491 every 8 connection(s)
I20260806 01:01:19.120249 15819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:19.129244 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 15819
I20260806 01:01:19.129681 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 14931
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978079, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.188@KRBTEST.COM
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978079, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.189@KRBTEST.COM
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978079, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:19.156152 15940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.125059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:50313 (local address 127.5.108.188:34699)
0806 01:01:19.125331 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:19.125340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:19.127602 (+ 2262us) server_negotiation.cc:407] Connection header received
0806 01:01:19.130479 (+ 2877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:19.130485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:19.130577 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:19.130661 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:19.133250 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.134711 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.139420 (+ 4709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.140257 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.140375 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.146451 (+ 6076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:19.146459 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:19.146467 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:19.146515 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:19.151550 (+ 5035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.152723 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.152731 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.152738 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.153187 (+ 449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.153785 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.153793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.153796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.154231 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:19.154365 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:19.155457 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0806 01:01:19.155724 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 01:01:19.158835 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.123567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:53955 (local address 127.5.108.189:43679)
0806 01:01:19.137570 (+ 14003us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:19.137579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:19.138340 (+ 761us) server_negotiation.cc:407] Connection header received
0806 01:01:19.138505 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:19.138510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:19.138605 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:19.138727 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:19.139907 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.140843 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.142368 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.142769 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.142906 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.148147 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:19.148155 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:19.148159 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:19.148194 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:19.151906 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.153072 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.153080 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.153086 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.153256 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.153809 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.153817 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.153823 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.154243 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:19.157883 (+ 3640us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:19.158030 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 01:01:19.158362 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13835,"thread_start_us":13972,"threads_started":1}
I20260806 01:01:19.159240 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.124354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:34699 (local address 127.5.108.129:50313)
0806 01:01:19.127440 (+ 3086us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:19.127499 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:19.130154 (+ 2655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:19.131237 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:19.131247 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:19.132162 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:19.133026 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.133052 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.135527 (+ 2475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.135536 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.138233 (+ 2697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.138253 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.140830 (+ 2577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.140840 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.141208 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.142406 (+ 1198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:19.142442 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:19.146146 (+ 3704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:19.151763 (+ 5617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.151776 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.151795 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.152571 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.153339 (+ 768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.153343 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.153348 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.153587 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.154468 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:19.154473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:19.154925 (+ 452us) client_negotiation.cc:769] Sending connection context
0806 01:01:19.156479 (+ 1554us) client_negotiation.cc:240] Negotiation successful
0806 01:01:19.156787 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2719,"mutex_wait_us":45,"thread_start_us":8696,"threads_started":1}
I20260806 01:01:19.159240 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.126263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.5.108.129:53955)
0806 01:01:19.138225 (+ 11962us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:19.138240 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:19.138379 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:19.138740 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:19.138745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:19.139450 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:19.139765 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.139775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.140992 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.141003 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.142227 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.142238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.142953 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.142959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.143135 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.144623 (+ 1488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:19.144651 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:19.147910 (+ 3259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:19.152037 (+ 4127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.152046 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.152050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.152927 (+ 877us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.153401 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.153409 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.153415 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.153649 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.154413 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:19.154425 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:19.154960 (+ 535us) client_negotiation.cc:769] Sending connection context
0806 01:01:19.155245 (+ 285us) client_negotiation.cc:240] Negotiation successful
0806 01:01:19.155873 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11867,"mutex_wait_us":6,"spinlock_wait_cycles":23808,"thread_start_us":621,"threads_started":1}
I20260806 01:01:19.160952 16156 heartbeater.cc:347] Connected to a master server at 127.5.108.188:34699
I20260806 01:01:19.161252 16156 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:19.161635 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.123518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:35879 (local address 127.5.108.190:46687)
0806 01:01:19.125685 (+ 2167us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:19.125693 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:19.134860 (+ 9167us) server_negotiation.cc:407] Connection header received
0806 01:01:19.134944 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:19.134952 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:19.135050 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:19.135190 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:19.143495 (+ 8305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.144321 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.147293 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.147976 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.148094 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.152036 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:19.152043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:19.152048 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:19.152091 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:19.155754 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.156617 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.156634 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.156637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.156838 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.157354 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.157362 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.157365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.157845 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:19.157979 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:19.159738 (+ 1759us) server_negotiation.cc:299] Negotiation successful
0806 01:01:19.160969 (+ 1231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1999,"thread_start_us":1870,"threads_started":1}
I20260806 01:01:19.162215 16156 heartbeater.cc:511] Master 127.5.108.188:34699 requested a full tablet report, sending...
I20260806 01:01:19.162845 16164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.125562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.129:35879)
0806 01:01:19.134033 (+ 8471us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:19.134056 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:19.134241 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:19.138425 (+ 4184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:19.138433 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:19.141702 (+ 3269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:19.142106 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.142121 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.146000 (+ 3879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.146005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.147153 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.147165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.148327 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.148333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.148513 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.149055 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:19.149093 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:19.151766 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:19.155905 (+ 4139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.155910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.155914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.156489 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.156990 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.156994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.156998 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.157178 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.158044 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:19.158047 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:19.159313 (+ 1266us) client_negotiation.cc:769] Sending connection context
0806 01:01:19.161939 (+ 2626us) client_negotiation.cc:240] Negotiation successful
0806 01:01:19.162282 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8310,"thread_start_us":12869,"threads_started":1}
I20260806 01:01:19.163201 16158 heartbeater.cc:347] Connected to a master server at 127.5.108.189:43679
I20260806 01:01:19.163410 16158 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:19.164032 16158 heartbeater.cc:511] Master 127.5.108.189:43679 requested a full tablet report, sending...
I20260806 01:01:19.164983 15863 ts_manager.cc:194] Registered new tserver with Master: e659777d823d41cc976196dc95dd78c3 (127.5.108.129:35491)
I20260806 01:01:19.165269 16157 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46687
I20260806 01:01:19.165439 16157 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:19.165920 16157 heartbeater.cc:511] Master 127.5.108.190:46687 requested a full tablet report, sending...
I20260806 01:01:19.166100 15552 ts_manager.cc:194] Registered new tserver with Master: e659777d823d41cc976196dc95dd78c3 (127.5.108.129:35491)
I20260806 01:01:19.167089 14629 ts_manager.cc:194] Re-registered known tserver with Master: e659777d823d41cc976196dc95dd78c3 (127.5.108.129:35491)
I20260806 01:01:19.168067 14629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:35879
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.130@KRBTEST.COM
I20260806 01:01:19.179163 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.131142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53610 (local address 127.5.108.130:44927)
0806 01:01:19.158019 (+ 26877us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:19.158027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:19.158779 (+ 752us) server_negotiation.cc:407] Connection header received
0806 01:01:19.159095 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:19.159107 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:19.159410 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:19.159555 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:19.162670 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.164490 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.166719 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.167673 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.167808 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.173083 (+ 5275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:19.173093 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:19.173104 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:19.173178 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:19.176281 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.176975 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.176984 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.176987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.177164 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.177524 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.177529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.177533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.177837 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:19.178133 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:19.178431 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 01:01:19.178746 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26450,"thread_start_us":26596,"threads_started":1}
I20260806 01:01:19.180513 15112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:19.264424 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 14931
I20260806 01:01:19.290273 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:44927
--local_ip_for_outbound_sockets=127.5.108.130
--tserver_master_addrs=127.5.108.190:46687
--webserver_port=38801
--webserver_interface=127.5.108.130
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab
--principal=oryx/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.108.190:46687,127.5.108.189:43679
--tserver_master_addrs=127.5.108.190:46687,127.5.108.189:43679,127.5.108.188:34699 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:01:19.478890 15718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: NON_VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: true } } }
I20260806 01:01:19.479310 15718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:19.480795 15718 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: attempting to promote NON_VOTER a9a9b8f63bc94143a5271806507d1c15 to VOTER
I20260806 01:01:19.481667 15718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } }
I20260806 01:01:19.484018 15882 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 LEARNER]: Refusing update from remote peer 23def1fd01f54c93ac66ee4d7c22c8c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 01:01:19.484413 15567 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Refusing update from remote peer 23def1fd01f54c93ac66ee4d7c22c8c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 01:01:19.484778 15718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 01:01:19.485126 16177 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 01:01:19.488162 16177 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a9a9b8f63bc94143a5271806507d1c15 (127.5.108.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:19.488749 15567 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a9a9b8f63bc94143a5271806507d1c15 (127.5.108.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:19.491832 16177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:19.492079 16177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:19.492316 16177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:19.492506 16177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:19.495275 16185 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:19.495723 16185 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:01:19.499334 16186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:19.499775 16186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:19.500085 15882 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a9a9b8f63bc94143a5271806507d1c15 (127.5.108.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:19.503335 16190 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:19.503861 16190 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 01:01:19.622201 16173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:19.622509 16173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:19.622689 16173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:19.629776 16173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:19.629907 16173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:19.630061 16173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:19.639127 16173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab
--principal=oryx/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:44927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=38801
--tserver_master_addrs=127.5.108.190:46687,127.5.108.189:43679,127.5.108.188:34699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:19.640436 16173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:19.641819 16173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:19.654575 16195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:19.654543 16198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:19.654549 16196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:19.655639 16173 server_base.cc:1034] running on GCE node
I20260806 01:01:19.656268 16173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:19.657142 16173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:19.658341 16173 hybrid_clock.cc:648] HybridClock initialized: now 1785978079658291 us; error 43 us; skew 500 ppm
I20260806 01:01:19.658720 16173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978079, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:19.661738 16173 init.cc:377] Logged in from keytab as oryx/127.5.108.130@KRBTEST.COM (short username oryx)
I20260806 01:01:19.663307 16173 webserver.cc:460] Webserver started at http://127.5.108.130:38801/ using document root <none> and password file <none>
I20260806 01:01:19.663988 16173 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:19.664103 16173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:19.669346 16173 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 01:01:19.672209 16210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:19.673457 16173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:19.673632 16173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "aad0aac9bfef4ac5883682d342760f17"
format_stamp: "Formatted at 2026-08-06 01:01:16 on dist-test-slave-x4qh"
server_key: "f40e15c58f5fa9d30624da1bfb969f8e"
server_key_iv: "498fd6f7e1dccc2ddece1bef462e3232"
server_key_version: "encryptionkey@0"
I20260806 01:01:19.674559 16173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:19.688359 16173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:19.692065 16173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:19.692329 16173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:19.693188 16173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:19.694427 16173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:19.694550 16173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:19.694664 16173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:19.694746 16173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:19.728871 16173 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:44927
I20260806 01:01:19.729054 16381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:44927 every 8 connection(s)
I20260806 01:01:19.730367 16173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:19.734421 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 16173
I20260806 01:01:19.734917 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 15128
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.131@KRBTEST.COM
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978079, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.189@KRBTEST.COM
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978079, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.188@KRBTEST.COM
Aug 06 01:01:19 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978079, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:19.775355 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.734692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:52733 (local address 127.5.108.189:43679)
0806 01:01:19.735434 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:19.735443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:19.745938 (+ 10495us) server_negotiation.cc:407] Connection header received
0806 01:01:19.751237 (+ 5299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:19.751243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:19.751408 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:19.751585 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:19.754639 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.756050 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.759529 (+ 3479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.760429 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.760531 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.766974 (+ 6443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:19.766982 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:19.766986 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:19.767026 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:19.771888 (+ 4862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.772627 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.772632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.772636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.772788 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.773289 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.773294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.773298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.773679 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:19.773831 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:19.774573 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 01:01:19.774787 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":421,"threads_started":1}
I20260806 01:01:19.776592 16392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.734220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.5.108.130:52733)
0806 01:01:19.745378 (+ 11158us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:19.745450 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:19.750831 (+ 5381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:19.751810 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:19.751822 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:19.753064 (+ 1242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:19.754428 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.754450 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.756286 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.756292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.759335 (+ 3043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.759344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.760950 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.760958 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.761403 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.762527 (+ 1124us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:19.762547 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:19.766703 (+ 4156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:19.772020 (+ 5317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.772028 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.772042 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.772500 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.772911 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.772915 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.772919 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.773165 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.773823 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:19.773830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:19.774172 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 01:01:19.774435 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 01:01:19.774963 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8644,"mutex_wait_us":368,"spinlock_wait_cycles":45824,"thread_start_us":15306,"threads_started":1}
I20260806 01:01:19.776947 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.736786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55032 (local address 127.5.108.131:45995)
0806 01:01:19.746555 (+ 9769us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:19.746567 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:19.746605 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:01:19.746696 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:19.746707 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:19.747006 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:19.747209 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:19.750330 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.751866 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.753867 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.754841 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.760851 (+ 6010us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.763352 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:19.763367 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:19.763419 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:19.763485 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:19.768686 (+ 5201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.770736 (+ 2050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.770746 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.770752 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.770993 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.773841 (+ 2848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.773848 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.773854 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.774342 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:19.774718 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:19.776118 (+ 1400us) server_negotiation.cc:299] Negotiation successful
0806 01:01:19.776409 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9506,"thread_start_us":9292,"threads_started":1}
I20260806 01:01:19.778138 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.733649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:34109 (local address 127.5.108.188:34699)
0806 01:01:19.734546 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:19.734553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:19.750169 (+ 15616us) server_negotiation.cc:407] Connection header received
0806 01:01:19.751142 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:19.751147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:19.751234 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:19.751331 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:19.754650 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.756020 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.759529 (+ 3509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.760446 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.760531 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.768210 (+ 7679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:19.768221 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:19.768227 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:19.768270 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:19.773027 (+ 4757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.774202 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.774208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.774211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.774432 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.775356 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.775361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.775364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.775789 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:19.775921 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:19.777471 (+ 1550us) server_negotiation.cc:299] Negotiation successful
0806 01:01:19.777737 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":1567,"threads_started":1}
I20260806 01:01:19.778461 16386 heartbeater.cc:347] Connected to a master server at 127.5.108.189:43679
I20260806 01:01:19.778580 15326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:19.778839 16386 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:19.779964 16386 heartbeater.cc:511] Master 127.5.108.189:43679 requested a full tablet report, sending...
I20260806 01:01:19.780884 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.734667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.130:48779)
0806 01:01:19.749110 (+ 14443us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:19.749128 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:19.750836 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:19.751810 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:19.751822 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:19.753053 (+ 1231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:19.754455 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.754470 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.756294 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.756299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.759462 (+ 3163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.759480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.761801 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.761806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.762109 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.763481 (+ 1372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:19.763502 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:19.768570 (+ 5068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:19.773320 (+ 4750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.773324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.773327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.775190 (+ 1863us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.776858 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.776864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.776867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.777104 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.779596 (+ 2492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:19.779601 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:19.779828 (+ 227us) client_negotiation.cc:769] Sending connection context
0806 01:01:19.780070 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 01:01:19.780347 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14288,"mutex_wait_us":304,"thread_start_us":1819,"threads_started":1}
I20260806 01:01:19.781247 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.734163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:48779 (local address 127.5.108.190:46687)
0806 01:01:19.744529 (+ 10366us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:19.744538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:19.749221 (+ 4683us) server_negotiation.cc:407] Connection header received
0806 01:01:19.751152 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:19.751158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:19.751247 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:19.751332 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:19.754640 (+ 3308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.756059 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.759654 (+ 3595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.760648 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.760789 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.768818 (+ 8029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:19.768824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:19.768828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:19.768876 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:19.773169 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.776484 (+ 3315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.776489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.776493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.776697 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:19.778958 (+ 2261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:19.778963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:19.778966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:19.779323 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:19.779493 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:19.780256 (+ 763us) server_negotiation.cc:299] Negotiation successful
0806 01:01:19.780562 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10195,"thread_start_us":10357,"threads_started":1}
I20260806 01:01:19.781376 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:19.736034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:34699 (local address 127.5.108.130:34109)
0806 01:01:19.750048 (+ 14014us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:19.750067 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:19.750831 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:19.751810 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:19.751822 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:19.753196 (+ 1374us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:19.754423 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.754450 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.756296 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.756299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.759325 (+ 3026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:19.759340 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:19.760950 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:19.760958 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:19.761404 (+ 446us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:19.762496 (+ 1092us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:19.762535 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:19.767877 (+ 5342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:19.773210 (+ 5333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.773217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.773223 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.774051 (+ 828us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.774604 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:19.774610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:19.774615 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:19.774806 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:19.776021 (+ 1215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:19.776025 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:19.776217 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 01:01:19.778324 (+ 2107us) client_negotiation.cc:240] Negotiation successful
0806 01:01:19.778658 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13829,"mutex_wait_us":170,"spinlock_wait_cycles":19840,"thread_start_us":12258,"threads_started":1}
I20260806 01:01:19.782595 15552 ts_manager.cc:194] Registered new tserver with Master: aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927)
I20260806 01:01:19.783558 16384 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46687
I20260806 01:01:19.783696 16384 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:19.784644 16384 heartbeater.cc:511] Master 127.5.108.190:46687 requested a full tablet report, sending...
I20260806 01:01:19.785288 16383 heartbeater.cc:347] Connected to a master server at 127.5.108.188:34699
I20260806 01:01:19.785434 16383 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:19.785773 14629 ts_manager.cc:194] Re-registered known tserver with Master: aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927)
I20260806 01:01:19.785939 16383 heartbeater.cc:511] Master 127.5.108.188:34699 requested a full tablet report, sending...
I20260806 01:01:19.786928 14629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:48779
I20260806 01:01:19.787446 15863 ts_manager.cc:194] Registered new tserver with Master: aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927)
I20260806 01:01:19.864740 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 15128
I20260806 01:01:19.889611 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:45995
--local_ip_for_outbound_sockets=127.5.108.131
--tserver_master_addrs=127.5.108.190:46687
--webserver_port=41869
--webserver_interface=127.5.108.131
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab
--principal=oryx/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.108.190:46687,127.5.108.189:43679
--tserver_master_addrs=127.5.108.190:46687,127.5.108.189:43679,127.5.108.188:34699 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:01:20.170358 16157 heartbeater.cc:503] Master 127.5.108.190:46687 was elected leader, sending a full tablet report...
W20260806 01:01:20.225550 16398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:20.225809 16398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:20.225900 16398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:20.232697 16398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:20.232788 16398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:20.232890 16398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:01:20.241693 16398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab
--principal=oryx/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:45995
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=41869
--tserver_master_addrs=127.5.108.190:46687,127.5.108.189:43679,127.5.108.188:34699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:20.242828 16398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:20.244061 16398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:20.257570 16411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:20.257614 16415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:20.257484 16410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:20.258553 16398 server_base.cc:1034] running on GCE node
I20260806 01:01:20.259260 16398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:20.260200 16398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:20.261432 16398 hybrid_clock.cc:648] HybridClock initialized: now 1785978080261394 us; error 39 us; skew 500 ppm
I20260806 01:01:20.261758 16398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:20 dist-test-slave-x4qh krb5kdc[14562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978080, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:20.265148 16398 init.cc:377] Logged in from keytab as oryx/127.5.108.131@KRBTEST.COM (short username oryx)
I20260806 01:01:20.266539 16398 webserver.cc:460] Webserver started at http://127.5.108.131:41869/ using document root <none> and password file <none>
I20260806 01:01:20.267233 16398 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:20.267441 16398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:20.272751 16398 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 01:01:20.275687 16424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:20.277115 16398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:01:20.277256 16398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "7d6e509992994b87a41b46c457c8d74b"
format_stamp: "Formatted at 2026-08-06 01:01:16 on dist-test-slave-x4qh"
server_key: "540c28c2d9298a29a01ed31f95b40c64"
server_key_iv: "3ca7ceec7676fb9b69cafbbe2183774b"
server_key_version: "encryptionkey@0"
I20260806 01:01:20.278760 16398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:20.303671 16398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:20.307832 16398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:20.308079 16398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:20.309180 16398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:20.310571 16398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:20.310704 16398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:20.310868 16398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:20.310961 16398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:20.346558 16398 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:45995
I20260806 01:01:20.346652 16580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:45995 every 8 connection(s)
I20260806 01:01:20.348140 16398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:20.355978 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 16398
Aug 06 01:01:20 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978080, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.189@KRBTEST.COM
Aug 06 01:01:20 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978080, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
Aug 06 01:01:20 dist-test-slave-x4qh krb5kdc[14562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978080, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.188@KRBTEST.COM
I20260806 01:01:20.407929 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.352612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:41303 (local address 127.5.108.188:34699)
0806 01:01:20.372379 (+ 19767us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.372387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.372407 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:01:20.372473 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.372478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.372556 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.373643 (+ 1087us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:20.377627 (+ 3984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.378879 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.380966 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.381617 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.381730 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.393818 (+ 12088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:20.393825 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:20.393829 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:20.393871 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:20.399017 (+ 5146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.401278 (+ 2261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.401283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.401288 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.401503 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.402934 (+ 1431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.402940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.402945 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.403473 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:20.403655 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.407274 (+ 3619us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.407507 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19629,"thread_start_us":19526,"threads_started":1}
I20260806 01:01:20.408504 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.351777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:34699 (local address 127.5.108.131:41303)
0806 01:01:20.360291 (+ 8514us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:20.360358 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:20.363350 (+ 2992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:20.373546 (+ 10196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:20.373552 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:20.375532 (+ 1980us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:20.375836 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.375861 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.379052 (+ 3191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.379058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.380792 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.380806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.382218 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.382228 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.383786 (+ 1558us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.384834 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:20.385092 (+ 258us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:20.393563 (+ 8471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:20.400262 (+ 6699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:20.400275 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:20.400302 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:20.401073 (+ 771us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:20.402447 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:20.402452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:20.402457 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:20.402797 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:20.403748 (+ 951us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:20.403757 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:20.404289 (+ 532us) client_negotiation.cc:769] Sending connection context
0806 01:01:20.404715 (+ 426us) client_negotiation.cc:240] Negotiation successful
0806 01:01:20.405317 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8209,"mutex_wait_us":1,"thread_start_us":7351,"threads_started":1}
I20260806 01:01:20.410365 16584 heartbeater.cc:347] Connected to a master server at 127.5.108.188:34699
I20260806 01:01:20.410640 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.351173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:58437 (local address 127.5.108.189:43679)
0806 01:01:20.372586 (+ 21413us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.372595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.372615 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:01:20.372679 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.372683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.372772 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.372893 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:20.376026 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.377188 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.382460 (+ 5272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.383186 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.383347 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.390394 (+ 7047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:20.390401 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:20.390408 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:20.390459 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:20.402684 (+ 12225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.404784 (+ 2100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.404789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.404791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.404965 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.405432 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.405437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.405440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.405813 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:20.407830 (+ 2017us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.409879 (+ 2049us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.410091 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21292,"thread_start_us":21168,"threads_started":1}
I20260806 01:01:20.410732 16584 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:20.412109 16584 heartbeater.cc:511] Master 127.5.108.188:34699 requested a full tablet report, sending...
I20260806 01:01:20.412957 16592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.352927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:43293 (local address 127.5.108.190:46687)
0806 01:01:20.359698 (+ 6771us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.359705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.371983 (+ 12278us) server_negotiation.cc:407] Connection header received
0806 01:01:20.373485 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.373494 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.373576 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.373676 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:20.376378 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.377650 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.382522 (+ 4872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.383262 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.383347 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.390746 (+ 7399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:20.390753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:20.390758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:20.390800 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:20.402272 (+ 11472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.404606 (+ 2334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.404627 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.404632 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.404916 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.407384 (+ 2468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.407390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.407394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.407914 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:20.408080 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.409087 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.412426 (+ 3339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6647,"thread_start_us":7176,"threads_started":1}
I20260806 01:01:20.414053 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.351777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.5.108.131:58437)
0806 01:01:20.366985 (+ 15208us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:20.367007 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:20.367143 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:20.373157 (+ 6014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:20.373172 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:20.374824 (+ 1652us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:20.375836 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.375861 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.377947 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.377953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.382323 (+ 4370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.382337 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.383748 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.383756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.383932 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.384956 (+ 1024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:20.384995 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:20.390120 (+ 5125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:20.402880 (+ 12760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:20.402887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:20.402890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:20.403647 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:20.405117 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:20.405121 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:20.405124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:20.405298 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:20.407910 (+ 2612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:20.407916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:20.409485 (+ 1569us) client_negotiation.cc:769] Sending connection context
0806 01:01:20.411077 (+ 1592us) client_negotiation.cc:240] Negotiation successful
0806 01:01:20.411424 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15029,"mutex_wait_us":1611,"spinlock_wait_cycles":52608,"thread_start_us":15441,"threads_started":1}
I20260806 01:01:20.414191 15863 ts_manager.cc:194] Registered new tserver with Master: 7d6e509992994b87a41b46c457c8d74b (127.5.108.131:45995)
I20260806 01:01:20.414623 16593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.353591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.131:43293)
0806 01:01:20.371861 (+ 18270us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:20.371883 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:20.372753 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:20.373848 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:20.373852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:20.375123 (+ 1271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:20.376205 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.376215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.379020 (+ 2805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.379027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.381982 (+ 2955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.381996 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.384309 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.384316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.384563 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.386994 (+ 2431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:20.387018 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:20.390479 (+ 3461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:20.403782 (+ 13303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:20.403785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:20.403788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:20.404400 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:20.406835 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:20.406842 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:20.406855 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:20.407176 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:20.408172 (+ 996us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:20.408176 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:20.408554 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 01:01:20.413203 (+ 4649us) client_negotiation.cc:240] Negotiation successful
0806 01:01:20.413485 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18120,"mutex_wait_us":691,"thread_start_us":17985,"threads_started":1}
I20260806 01:01:20.415541 16586 heartbeater.cc:347] Connected to a master server at 127.5.108.189:43679
I20260806 01:01:20.415681 16586 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:20.416556 16585 heartbeater.cc:347] Connected to a master server at 127.5.108.190:46687
I20260806 01:01:20.416733 16585 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:20.417433 16585 heartbeater.cc:511] Master 127.5.108.190:46687 requested a full tablet report, sending...
I20260806 01:01:20.417968 16586 heartbeater.cc:511] Master 127.5.108.189:43679 requested a full tablet report, sending...
I20260806 01:01:20.419556 14629 ts_manager.cc:194] Re-registered known tserver with Master: 7d6e509992994b87a41b46c457c8d74b (127.5.108.131:45995)
I20260806 01:01:20.419544 15552 ts_manager.cc:194] Registered new tserver with Master: 7d6e509992994b87a41b46c457c8d74b (127.5.108.131:45995)
I20260806 01:01:20.420852 14629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:43293
I20260806 01:01:20.429785 16602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.377619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40466 (local address 127.5.108.190:46687)
0806 01:01:20.400618 (+ 22999us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.400624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.400642 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:01:20.400694 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.400698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.400771 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.400904 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:20.404615 (+ 3711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.406662 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.410513 (+ 3851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.411415 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.411571 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.421255 (+ 9684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:20.421264 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:20.421270 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:20.421316 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:20.425730 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.426888 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.426894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.426898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.427139 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.427637 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.427643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.427647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.428203 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:20.428482 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.428971 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.429278 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22849,"thread_start_us":22714,"threads_started":1}
I20260806 01:01:20.430860 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.383914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50698 (local address 127.5.108.189:43679)
0806 01:01:20.395427 (+ 11513us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.395434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.397622 (+ 2188us) server_negotiation.cc:407] Connection header received
0806 01:01:20.398458 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.398486 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.398597 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.398726 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:20.406024 (+ 7298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.407676 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.416208 (+ 8532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.417180 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.417306 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.423258 (+ 5952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:20.423263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:20.423266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:20.423298 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:20.427019 (+ 3721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.427998 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.428002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.428005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.428173 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.428621 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.428626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.428631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.429044 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:20.429502 (+ 458us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.430085 (+ 583us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.430314 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11320,"thread_start_us":11112,"threads_started":1}
I20260806 01:01:20.432981 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.383071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.5.108.188:34699)
0806 01:01:20.396002 (+ 12931us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.396008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.398247 (+ 2239us) server_negotiation.cc:407] Connection header received
0806 01:01:20.399274 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.399282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.399459 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.399575 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:20.406136 (+ 6561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.407941 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.414277 (+ 6336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.415511 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.415662 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.424469 (+ 8807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:20.424486 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:20.424491 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:20.424530 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:20.429455 (+ 4925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.430700 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.430706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.430709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.430891 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.431436 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.431444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.431450 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.431818 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:20.432001 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.432533 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.432730 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12783,"thread_start_us":12577,"threads_started":1}
I20260806 01:01:20.435869 14629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:01:20.442842 14629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:01:20.484138 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.463540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.5.108.131:45995)
0806 01:01:20.473734 (+ 10194us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.473746 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.474535 (+ 789us) server_negotiation.cc:407] Connection header received
0806 01:01:20.475334 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.475347 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.475708 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.475962 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:20.476928 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.478566 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.481335 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.482926 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.483087 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.483312 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.483633 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.483814 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9964,"thread_start_us":9748,"threads_started":1}
I20260806 01:01:20.484189 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.462882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:45995 (local address 127.0.0.1:45158)
0806 01:01:20.474380 (+ 11498us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:20.474405 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:20.474579 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:20.476299 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:20.476303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:20.476332 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:20.476690 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.476698 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.478744 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.478747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.481142 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.481158 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.483178 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.483188 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.483362 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.483446 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 01:01:20.483561 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 01:01:20.483706 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11328,"thread_start_us":13277,"threads_started":1}
I20260806 01:01:20.485497 16621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.462639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40532 (local address 127.5.108.129:35491)
0806 01:01:20.477489 (+ 14850us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.477498 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.477525 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:01:20.477625 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.477634 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.477850 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.478221 (+ 371us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:20.478689 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.480330 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.482617 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.484219 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.484369 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.484573 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.484955 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.485120 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14681,"thread_start_us":14571,"threads_started":1}
I20260806 01:01:20.485597 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.462287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35491 (local address 127.0.0.1:40532)
0806 01:01:20.472067 (+ 9780us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:20.472108 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:20.472319 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:20.478108 (+ 5789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:20.478113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:20.478139 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:20.478499 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.478511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.480468 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.480473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.482428 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.482442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.484616 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.484624 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.484772 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.484796 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:01:20.484910 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 01:01:20.485050 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9555,"mutex_wait_us":72,"thread_start_us":9361,"threads_started":1}
I20260806 01:01:20.485899 16628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.464067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.5.108.130:44927)
0806 01:01:20.473043 (+ 8976us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.473058 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.476790 (+ 3732us) server_negotiation.cc:407] Connection header received
0806 01:01:20.477034 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.477041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.477281 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.477505 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:20.478187 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.479792 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.483204 (+ 3412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.484563 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.484656 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.484765 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.485412 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.485597 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8743,"thread_start_us":8593,"threads_started":1}
I20260806 01:01:20.486768 16622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.463393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44927 (local address 127.0.0.1:33510)
0806 01:01:20.476621 (+ 13228us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:20.476656 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:20.476838 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:20.477495 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:20.477500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:20.477529 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:20.477944 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.477959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.479955 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.479960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.483013 (+ 3053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.483029 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.485021 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.485025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.485183 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.485213 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:01:20.486179 (+ 966us) client_negotiation.cc:240] Negotiation successful
0806 01:01:20.486378 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13032,"mutex_wait_us":65,"thread_start_us":10597,"threads_started":1}
I20260806 01:01:20.489207 16043 tablet_service.cc:1467] Processing CreateTablet for tablet 762cf32726f044c68ec0e3d792fee9a5 (DEFAULT_TABLE table=test-table [id=4325e0a655724737b634540a71cb449b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:20.489194 16493 tablet_service.cc:1467] Processing CreateTablet for tablet 762cf32726f044c68ec0e3d792fee9a5 (DEFAULT_TABLE table=test-table [id=4325e0a655724737b634540a71cb449b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:20.490501 16043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762cf32726f044c68ec0e3d792fee9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:20.490517 16493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762cf32726f044c68ec0e3d792fee9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:20.495873 16285 tablet_service.cc:1467] Processing CreateTablet for tablet 762cf32726f044c68ec0e3d792fee9a5 (DEFAULT_TABLE table=test-table [id=4325e0a655724737b634540a71cb449b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:20.497268 16285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762cf32726f044c68ec0e3d792fee9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:20.504060 16629 tablet_bootstrap.cc:492] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: Bootstrap starting.
I20260806 01:01:20.508193 16629 tablet_bootstrap.cc:654] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:20.509034 16632 tablet_bootstrap.cc:492] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: Bootstrap starting.
I20260806 01:01:20.510728 16629 log.cc:824] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:20.513700 16632 tablet_bootstrap.cc:654] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:20.515393 16629 tablet_bootstrap.cc:492] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: No bootstrap required, opened a new log
I20260806 01:01:20.515345 16632 log.cc:824] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:20.515815 16629 ts_tablet_manager.cc:1397] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260806 01:01:20.516012 16634 tablet_bootstrap.cc:492] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: Bootstrap starting.
I20260806 01:01:20.519784 16634 tablet_bootstrap.cc:654] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:20.521119 16634 log.cc:824] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:20.521495 16632 tablet_bootstrap.cc:492] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: No bootstrap required, opened a new log
I20260806 01:01:20.521853 16632 ts_tablet_manager.cc:1397] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260806 01:01:20.524067 16634 tablet_bootstrap.cc:492] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: No bootstrap required, opened a new log
I20260806 01:01:20.524371 16634 ts_tablet_manager.cc:1397] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 01:01:20.529994 16629 raft_consensus.cc:348] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:20.530561 16629 raft_consensus.cc:374] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:20.530771 16629 raft_consensus.cc:729] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6e509992994b87a41b46c457c8d74b, State: Initialized, Role: FOLLOWER
I20260806 01:01:20.531612 16629 consensus_queue.cc:260] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:20.534040 16629 ts_tablet_manager.cc:1428] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: Time spent starting tablet: real 0.018s user 0.011s sys 0.006s
I20260806 01:01:20.534492 16585 heartbeater.cc:503] Master 127.5.108.190:46687 was elected leader, sending a full tablet report...
I20260806 01:01:20.535411 16634 raft_consensus.cc:348] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:20.536013 16634 raft_consensus.cc:374] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:20.536135 16634 raft_consensus.cc:729] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad0aac9bfef4ac5883682d342760f17, State: Initialized, Role: FOLLOWER
I20260806 01:01:20.536845 16634 consensus_queue.cc:260] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:20.536927 16632 raft_consensus.cc:348] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:20.537392 16632 raft_consensus.cc:374] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:20.537497 16632 raft_consensus.cc:729] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e659777d823d41cc976196dc95dd78c3, State: Initialized, Role: FOLLOWER
I20260806 01:01:20.538148 16632 consensus_queue.cc:260] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:20.543874 16632 ts_tablet_manager.cc:1428] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: Time spent starting tablet: real 0.022s user 0.022s sys 0.000s
I20260806 01:01:20.544790 16384 heartbeater.cc:503] Master 127.5.108.190:46687 was elected leader, sending a full tablet report...
I20260806 01:01:20.545732 16634 ts_tablet_manager.cc:1428] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: Time spent starting tablet: real 0.021s user 0.009s sys 0.010s
W20260806 01:01:20.606667 16587 tablet.cc:2369] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:01:20.629387 16160 tablet.cc:2369] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:01:20.668952 16642 raft_consensus.cc:482] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:20.669283 16642 raft_consensus.cc:504] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:20.671319 16642 leader_election.cc:288] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d6e509992994b87a41b46c457c8d74b (127.5.108.131:45995), aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927)
I20260806 01:01:20.680195 16628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.671816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:38735 (local address 127.5.108.130:44927)
0806 01:01:20.672004 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.672010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.672834 (+ 824us) server_negotiation.cc:407] Connection header received
0806 01:01:20.673158 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.673164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.673261 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.673351 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:20.673947 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.675021 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.678037 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.679032 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.679206 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.679303 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.679747 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.679896 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:01:20.680306 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.672265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:44647 (local address 127.5.108.131:45995)
0806 01:01:20.672477 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.672484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.673161 (+ 677us) server_negotiation.cc:407] Connection header received
0806 01:01:20.673317 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.673322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.673392 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.673471 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:20.674147 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.674838 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.677999 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.679248 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.679479 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.679555 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.679879 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.680070 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:01:20.680374 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.671658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44927 (local address 127.5.108.129:38735)
0806 01:01:20.672718 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:20.672736 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:20.672972 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:20.673378 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:20.673382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:20.673408 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:20.673777 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.673789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.675176 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.675184 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.677805 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.677820 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.679270 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.679277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.679555 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.679591 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:01:20.679685 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 01:01:20.679819 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":11,"thread_start_us":652,"threads_started":1}
I20260806 01:01:20.681116 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.671920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:45995 (local address 127.5.108.129:44647)
0806 01:01:20.673021 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:20.673059 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:20.673196 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:20.673551 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:20.673559 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:20.673578 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:20.673963 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.673972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.674984 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.674989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.677805 (+ 2816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:20.677820 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.679538 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.679544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:20.679699 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.679725 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 01:01:20.680668 (+ 943us) client_negotiation.cc:240] Negotiation successful
0806 01:01:20.680804 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"mutex_wait_us":113,"thread_start_us":783,"threads_started":1}
I20260806 01:01:20.681667 16315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "762cf32726f044c68ec0e3d792fee9a5" candidate_uuid: "e659777d823d41cc976196dc95dd78c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad0aac9bfef4ac5883682d342760f17" is_pre_election: true
I20260806 01:01:20.682093 16519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "762cf32726f044c68ec0e3d792fee9a5" candidate_uuid: "e659777d823d41cc976196dc95dd78c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6e509992994b87a41b46c457c8d74b" is_pre_election: true
I20260806 01:01:20.682297 16315 raft_consensus.cc:2454] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e659777d823d41cc976196dc95dd78c3 in term 0.
I20260806 01:01:20.682650 16519 raft_consensus.cc:2454] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e659777d823d41cc976196dc95dd78c3 in term 0.
I20260806 01:01:20.683041 15970 leader_election.cc:302] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aad0aac9bfef4ac5883682d342760f17, e659777d823d41cc976196dc95dd78c3; no voters:
I20260806 01:01:20.683514 16642 raft_consensus.cc:2781] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:01:20.683717 16642 raft_consensus.cc:482] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:01:20.683852 16642 raft_consensus.cc:3037] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:20.686075 16642 raft_consensus.cc:504] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:20.686707 16642 leader_election.cc:288] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [CANDIDATE]: Term 1 election: Requested vote from peers 7d6e509992994b87a41b46c457c8d74b (127.5.108.131:45995), aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927)
I20260806 01:01:20.687157 16519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "762cf32726f044c68ec0e3d792fee9a5" candidate_uuid: "e659777d823d41cc976196dc95dd78c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6e509992994b87a41b46c457c8d74b"
I20260806 01:01:20.687332 16519 raft_consensus.cc:3037] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:20.687615 16315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "762cf32726f044c68ec0e3d792fee9a5" candidate_uuid: "e659777d823d41cc976196dc95dd78c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad0aac9bfef4ac5883682d342760f17"
I20260806 01:01:20.687819 16315 raft_consensus.cc:3037] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:20.689558 16519 raft_consensus.cc:2454] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e659777d823d41cc976196dc95dd78c3 in term 1.
I20260806 01:01:20.689934 15968 leader_election.cc:302] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6e509992994b87a41b46c457c8d74b, e659777d823d41cc976196dc95dd78c3; no voters:
I20260806 01:01:20.690163 16315 raft_consensus.cc:2454] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e659777d823d41cc976196dc95dd78c3 in term 1.
I20260806 01:01:20.690235 16642 raft_consensus.cc:2781] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:20.690937 16642 raft_consensus.cc:686] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 1 LEADER]: Becoming Leader. State: Replica: e659777d823d41cc976196dc95dd78c3, State: Running, Role: LEADER
I20260806 01:01:20.691524 16642 consensus_queue.cc:237] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:20.697180 14624 catalog_manager.cc:5495] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e659777d823d41cc976196dc95dd78c3 (127.5.108.129). New cstate: current_term: 1 leader_uuid: "e659777d823d41cc976196dc95dd78c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } health_report { overall_health: UNKNOWN } } }
W20260806 01:01:20.749197 16387 tablet.cc:2369] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:01:20.768846 16621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.763418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.5.108.129:35491)
0806 01:01:20.763629 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.763635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.763655 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:01:20.763859 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.763863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.763947 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.764103 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:20.764553 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.765266 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.766596 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.767017 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.767154 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.767699 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:20.768178 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:20.768402 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.768455 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.768534 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:01:20.777208 16519 raft_consensus.cc:1261] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [term 1 FOLLOWER]: Refusing update from remote peer e659777d823d41cc976196dc95dd78c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:20.777375 16315 raft_consensus.cc:1261] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [term 1 FOLLOWER]: Refusing update from remote peer e659777d823d41cc976196dc95dd78c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:20.778285 16648 consensus_queue.cc:1035] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:20.778692 16642 consensus_queue.cc:1035] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:20.790879 16654 mvcc.cc:205] Tried to move back new op lower bound from 7315366218853707776 to 7315366218516246528. Current Snapshot: MvccSnapshot[applied={T|T < 7315366218853707776}]
I20260806 01:01:20.797376 16656 mvcc.cc:205] Tried to move back new op lower bound from 7315366218853707776 to 7315366218516246528. Current Snapshot: MvccSnapshot[applied={T|T < 7315366218853707776}]
I20260806 01:01:20.815068 14629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:20.815471 14629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:20.823307 14629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23def1fd01f54c93ac66ee4d7c22c8c2: Sending DeleteTablet for 3 replicas of tablet 762cf32726f044c68ec0e3d792fee9a5
I20260806 01:01:20.824328 16043 tablet_service.cc:1514] Processing DeleteTablet for tablet 762cf32726f044c68ec0e3d792fee9a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:20 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:40532
I20260806 01:01:20.824970 16493 tablet_service.cc:1514] Processing DeleteTablet for tablet 762cf32726f044c68ec0e3d792fee9a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:20 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:45158
I20260806 01:01:20.825682 16285 tablet_service.cc:1514] Processing DeleteTablet for tablet 762cf32726f044c68ec0e3d792fee9a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:20 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:33510
I20260806 01:01:20.828922 16671 tablet_replica.cc:330] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: stopping tablet replica
I20260806 01:01:20.829520 16671 raft_consensus.cc:2229] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:20.830368 16671 raft_consensus.cc:2258] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:20.832149 16668 tablet_replica.cc:330] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: stopping tablet replica
I20260806 01:01:20.832739 16668 raft_consensus.cc:2229] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:01:20.835201 16671 ts_tablet_manager.cc:1905] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:20.836501 16668 raft_consensus.cc:2258] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:20.839028 16668 ts_tablet_manager.cc:1905] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:20.844668 16670 tablet_replica.cc:330] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: stopping tablet replica
I20260806 01:01:20.845166 16670 raft_consensus.cc:2229] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:20.845818 16670 raft_consensus.cc:2258] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:20.847081 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 14582
I20260806 01:01:20.852384 14674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:20.854887 16670 ts_tablet_manager.cc:1905] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:20.856771 16668 ts_tablet_manager.cc:1918] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:20.856977 16668 log.cc:1196] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/762cf32726f044c68ec0e3d792fee9a5
I20260806 01:01:20.857875 16668 ts_tablet_manager.cc:1939] T 762cf32726f044c68ec0e3d792fee9a5 P e659777d823d41cc976196dc95dd78c3: Deleting consensus metadata
I20260806 01:01:20.858847 16671 ts_tablet_manager.cc:1918] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:20.859257 16671 log.cc:1196] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/762cf32726f044c68ec0e3d792fee9a5
I20260806 01:01:20.860106 16671 ts_tablet_manager.cc:1939] T 762cf32726f044c68ec0e3d792fee9a5 P aad0aac9bfef4ac5883682d342760f17: Deleting consensus metadata
I20260806 01:01:20.871786 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.831843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.5.108.189:43679)
0806 01:01:20.832046 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.832052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.845396 (+ 13344us) server_negotiation.cc:407] Connection header received
0806 01:01:20.846062 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.846069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.846159 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.846256 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:20.851183 (+ 4927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.852614 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.856809 (+ 4195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.859124 (+ 2315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.859265 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.865036 (+ 5771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:20.865045 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:20.865050 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:20.865093 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:20.868712 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.869552 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.869558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.869562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.869731 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.870178 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.870184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.870188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.870555 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:20.870800 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.871192 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.871469 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:01:20.875510 16670 ts_tablet_manager.cc:1918] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:20.875766 16670 log.cc:1196] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/762cf32726f044c68ec0e3d792fee9a5
I20260806 01:01:20.876472 16670 ts_tablet_manager.cc:1939] T 762cf32726f044c68ec0e3d792fee9a5 P 7d6e509992994b87a41b46c457c8d74b: Deleting consensus metadata
I20260806 01:01:20.887465 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:20.845480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42138 (local address 127.5.108.188:34699)
0806 01:01:20.845714 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:20.845720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:20.871884 (+ 26164us) server_negotiation.cc:407] Connection header received
0806 01:01:20.872182 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:20.872188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:20.872283 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:20.872399 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:20.874415 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.875618 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.877719 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:20.878385 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:20.878487 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:20.881334 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:20.881340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:20.881350 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:20.881382 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:20.884594 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.885230 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.885235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.885239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.885386 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:20.885933 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:20.885939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:20.885942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:20.886323 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:20.886588 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:20.886885 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 01:01:20.887110 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:01:21.018271 14609 catalog_manager.cc:4841] TS aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927): tablet 762cf32726f044c68ec0e3d792fee9a5 (table test-table [id=4325e0a655724737b634540a71cb449b]) successfully deleted
I20260806 01:01:21.018980 14603 catalog_manager.cc:4841] TS e659777d823d41cc976196dc95dd78c3 (127.5.108.129:35491): tablet 762cf32726f044c68ec0e3d792fee9a5 (table test-table [id=4325e0a655724737b634540a71cb449b]) successfully deleted
I20260806 01:01:21.019044 14606 catalog_manager.cc:4841] TS 7d6e509992994b87a41b46c457c8d74b (127.5.108.131:45995): tablet 762cf32726f044c68ec0e3d792fee9a5 (table test-table [id=4325e0a655724737b634540a71cb449b]) successfully deleted
W20260806 01:01:21.019196 14603 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.108.129:35491, user_credentials={real_user=slave}} blocked reactor thread for 159098us
I20260806 01:01:21.020380 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 14582
I20260806 01:01:21.078130 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:21.054626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50726 (local address 127.5.108.189:43679)
0806 01:01:21.054854 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:21.054863 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:21.062212 (+ 7349us) server_negotiation.cc:407] Connection header received
0806 01:01:21.062372 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:21.062379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:21.062471 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:21.062583 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:21.064856 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:21.065970 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:21.067741 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:21.068132 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:21.068268 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:21.072020 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:21.072027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:21.072031 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:21.072070 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:21.075135 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:21.075961 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:21.075965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:21.075968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:21.076086 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:21.076449 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:21.076454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:21.076462 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:21.076855 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:21.077122 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:21.077503 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 01:01:21.077746 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:01:21.079939 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:21.055226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42144 (local address 127.5.108.188:34699)
0806 01:01:21.055445 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:21.055451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:21.062187 (+ 6736us) server_negotiation.cc:407] Connection header received
0806 01:01:21.062410 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:21.062414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:21.062483 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:21.062563 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:21.065065 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:21.065801 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:21.067696 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:21.068305 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:21.068426 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:21.072931 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:21.072938 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:21.072943 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:21.072984 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:21.076512 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:21.077249 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:21.077254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:21.077258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:21.077415 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:21.077860 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:21.077866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:21.077869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:21.078346 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:21.078568 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:21.079105 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 01:01:21.079709 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:01:21.802206 16683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:21.801314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.131:53483)
0806 01:01:21.801843 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:21.801985 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":277,"threads_started":1}
W20260806 01:01:21.802672 16585 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:46687 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
I20260806 01:01:21.811304 16684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:21.810493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.129:33161)
0806 01:01:21.810952 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:21.811084 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":234,"threads_started":1}
W20260806 01:01:21.811784 16157 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:46687 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
I20260806 01:01:21.811823 16685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:21.810994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.130:35777)
0806 01:01:21.811475 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:21.811617 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":215,"threads_started":1}
W20260806 01:01:21.812561 16384 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:46687 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
I20260806 01:01:22.323277 16686 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 23def1fd01f54c93ac66ee4d7c22c8c2)
I20260806 01:01:22.323573 16686 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } }
I20260806 01:01:22.323954 16687 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 23def1fd01f54c93ac66ee4d7c22c8c2)
I20260806 01:01:22.324204 16687 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } }
I20260806 01:01:22.324625 16686 leader_election.cc:288] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23def1fd01f54c93ac66ee4d7c22c8c2 (127.5.108.190:46687), bc5c4f325eaa4df3917d92ebc48d5c41 (127.5.108.189:43679)
I20260806 01:01:22.325964 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.324709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:40890)
0806 01:01:22.325401 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.325674 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":350,"threads_started":1}
W20260806 01:01:22.326334 15840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
I20260806 01:01:22.333231 16687 leader_election.cc:288] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23def1fd01f54c93ac66ee4d7c22c8c2 (127.5.108.190:46687), a9a9b8f63bc94143a5271806507d1c15 (127.5.108.188:34699)
I20260806 01:01:22.335355 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.324848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:43679 (local address 127.0.0.1:50736)
0806 01:01:22.325679 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.325697 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:22.325846 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:22.326337 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:22.326342 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:22.326374 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:22.326699 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.326714 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.328869 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.328879 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.331737 (+ 2858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.331759 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.334219 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.334228 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.334479 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.334534 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 01:01:22.334644 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 01:01:22.334835 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":545,"threads_started":1}
I20260806 01:01:22.336381 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.324998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50736 (local address 127.5.108.189:43679)
0806 01:01:22.325999 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:22.326005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:22.326025 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:01:22.326091 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:22.326096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:22.326166 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:22.326274 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:22.326962 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.328624 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.331972 (+ 3348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.333755 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.333913 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.334063 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:22.335644 (+ 1581us) server_negotiation.cc:299] Negotiation successful
0806 01:01:22.335851 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895,"thread_start_us":722,"threads_started":1}
I20260806 01:01:22.337055 15567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9a9b8f63bc94143a5271806507d1c15" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" is_pre_election: true
I20260806 01:01:22.337548 15567 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a9b8f63bc94143a5271806507d1c15 in term 1.
I20260806 01:01:22.338201 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.326961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:40902)
0806 01:01:22.337599 (+ 10638us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.337835 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10479,"threads_started":1}
I20260806 01:01:22.338140 15841 leader_election.cc:302] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9a9b8f63bc94143a5271806507d1c15, bc5c4f325eaa4df3917d92ebc48d5c41; no voters:
I20260806 01:01:22.338551 16686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 01:01:22.338716 16686 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 23def1fd01f54c93ac66ee4d7c22c8c2)
I20260806 01:01:22.338843 16686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 1 FOLLOWER]: Advancing to term 2
W20260806 01:01:22.339063 15528 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
I20260806 01:01:22.341199 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.340783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:40930)
0806 01:01:22.340950 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.341034 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 01:01:22.341159 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.340520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:40916)
0806 01:01:22.340727 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.340916 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260806 01:01:22.341456 15528 leader_election.cc:334] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 23def1fd01f54c93ac66ee4d7c22c8c2 (127.5.108.190:46687): Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
W20260806 01:01:22.341745 15840 leader_election.cc:334] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 23def1fd01f54c93ac66ee4d7c22c8c2 (127.5.108.190:46687): Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
I20260806 01:01:22.342675 16686 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } }
I20260806 01:01:22.343415 16686 leader_election.cc:288] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [CANDIDATE]: Term 2 election: Requested vote from peers 23def1fd01f54c93ac66ee4d7c22c8c2 (127.5.108.190:46687), bc5c4f325eaa4df3917d92ebc48d5c41 (127.5.108.189:43679)
I20260806 01:01:22.343874 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.343460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:40944)
0806 01:01:22.343584 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.343685 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 01:01:22.343946 15567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9a9b8f63bc94143a5271806507d1c15" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41"
I20260806 01:01:22.344205 15567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:01:22.344969 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.344588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:40946)
0806 01:01:22.344760 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.344836 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 01:01:22.345168 15840 leader_election.cc:334] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 23def1fd01f54c93ac66ee4d7c22c8c2 (127.5.108.190:46687): Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
I20260806 01:01:22.347226 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.339269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.5.108.188:34699)
0806 01:01:22.340781 (+ 1512us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:22.340786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:22.340809 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:01:22.340877 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:22.340882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:22.340953 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:22.341715 (+ 762us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:22.342000 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.343218 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.345360 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.346168 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.346393 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.346504 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:22.346770 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 01:01:22.346929 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1365,"thread_start_us":1218,"threads_started":1}
I20260806 01:01:22.347222 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.327089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:34699 (local address 127.0.0.1:42146)
0806 01:01:22.338390 (+ 11301us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.338409 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:22.338545 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:22.341164 (+ 2619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:22.341169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:22.341236 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:22.341522 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.341557 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.343410 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.343416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.345167 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.345180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.346436 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.346442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.346621 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.346646 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 01:01:22.346746 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 01:01:22.346914 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11204,"thread_start_us":11073,"threads_started":1}
I20260806 01:01:22.347273 15567 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a9b8f63bc94143a5271806507d1c15 in term 2.
I20260806 01:01:22.347671 15841 leader_election.cc:302] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9a9b8f63bc94143a5271806507d1c15, bc5c4f325eaa4df3917d92ebc48d5c41; no voters: 23def1fd01f54c93ac66ee4d7c22c8c2
I20260806 01:01:22.348275 16686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:01:22.348977 16686 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [term 2 LEADER]: Becoming Leader. State: Replica: a9a9b8f63bc94143a5271806507d1c15, State: Running, Role: LEADER
I20260806 01:01:22.349489 16686 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } }
I20260806 01:01:22.349596 15879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a9a9b8f63bc94143a5271806507d1c15" is_pre_election: true
I20260806 01:01:22.350463 15532 leader_election.cc:302] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc5c4f325eaa4df3917d92ebc48d5c41; no voters: 23def1fd01f54c93ac66ee4d7c22c8c2, a9a9b8f63bc94143a5271806507d1c15
I20260806 01:01:22.350844 16687 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 01:01:22.352473 16705 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a9b8f63bc94143a5271806507d1c15. Latest consensus state: current_term: 2 leader_uuid: "a9a9b8f63bc94143a5271806507d1c15" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:22.352955 16705 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:22.353629 16708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:22.357754 16708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:22.358598 16708 catalog_manager.cc:1259] Loaded cluster ID: d3daf9e8901a4ca7a4ed8fcaeab76cb8
I20260806 01:01:22.358690 16708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:22.359892 16708 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:22.360653 16708 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Loaded TSK: 0
I20260806 01:01:22.361464 16708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:01:22.412338 15863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:01:22.419320 15863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:01:22.423133 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.422668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:40984)
0806 01:01:22.422877 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.422966 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 01:01:22.423228 15567 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [term 2 FOLLOWER]: Refusing update from remote peer a9a9b8f63bc94143a5271806507d1c15: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 01:01:22.424274 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.423839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:40992)
0806 01:01:22.424016 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.424104 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 01:01:22.424125 16705 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 01:01:22.424669 15840 consensus_peers.cc:487] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 -> Peer 23def1fd01f54c93ac66ee4d7c22c8c2 (127.5.108.190:46687): Couldn't send request to peer 23def1fd01f54c93ac66ee4d7c22c8c2. Status: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 01:01:22.426811 16687 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a9b8f63bc94143a5271806507d1c15. Latest consensus state: current_term: 2 leader_uuid: "a9a9b8f63bc94143a5271806507d1c15" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:22.427060 16687 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:01:22.428823 16705 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a9a9b8f63bc94143a5271806507d1c15" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:22.429122 16705 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:22.429530 16713 mvcc.cc:205] Tried to move back new op lower bound from 7315366225599262720 to 7315366225309519872. Current Snapshot: MvccSnapshot[applied={T|T < 7315366225599262720}]
I20260806 01:01:22.431922 16687 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a9a9b8f63bc94143a5271806507d1c15" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "23def1fd01f54c93ac66ee4d7c22c8c2" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46687 } } peers { permanent_uuid: "bc5c4f325eaa4df3917d92ebc48d5c41" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 43679 } attrs { promote: false } } peers { permanent_uuid: "a9a9b8f63bc94143a5271806507d1c15" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 34699 } attrs { promote: false } } }
I20260806 01:01:22.432243 16687 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c4f325eaa4df3917d92ebc48d5c41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:01:22.445914 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.437315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:45995 (local address 127.0.0.1:45164)
0806 01:01:22.437515 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.437538 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:22.437663 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:22.438590 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:22.438598 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:22.438635 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:22.439062 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.439077 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.441301 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.441308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.443521 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.443537 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.445159 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.445166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.445374 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.445412 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:01:22.445489 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 01:01:22.445634 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 01:01:22.446313 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.438359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.5.108.130:44927)
0806 01:01:22.439783 (+ 1424us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:22.439788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:22.439807 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:01:22.439875 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:22.439880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:22.439960 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:22.440059 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:22.440771 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.441823 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.444427 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.445295 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.445453 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.445572 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:22.445846 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 01:01:22.445982 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1279,"thread_start_us":1182,"threads_started":1}
I20260806 01:01:22.446506 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.438269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44927 (local address 127.0.0.1:33516)
0806 01:01:22.438924 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.438943 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:22.439053 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:22.440240 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:22.440245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:22.440287 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:22.440618 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.440626 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.441984 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.441989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.444235 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.444249 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.445525 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.445531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.445686 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.445713 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 01:01:22.445819 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 01:01:22.445944 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":28,"thread_start_us":328,"threads_started":1}
I20260806 01:01:22.446763 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.437649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45164 (local address 127.5.108.131:45995)
0806 01:01:22.438225 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:22.438232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:22.438251 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:01:22.438322 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:22.438327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:22.438410 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:22.439283 (+ 873us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:22.439730 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.441163 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.443708 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.444892 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.446159 (+ 1267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.446294 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:22.446346 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 01:01:22.446485 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":257,"threads_started":1}
I20260806 01:01:22.447033 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.436822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35491 (local address 127.0.0.1:40552)
0806 01:01:22.436954 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.436974 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:22.437109 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:22.441391 (+ 4282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:22.441396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:22.441428 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:22.441690 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.441701 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.442736 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.442741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.444507 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.444521 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.446333 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.446339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.446534 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.446567 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:01:22.446645 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 01:01:22.446762 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":42}
I20260806 01:01:22.447616 16493 tablet_service.cc:1467] Processing CreateTablet for tablet 4f733400846a4489b68d717902262020 (DEFAULT_TABLE table=test-table [id=bfdaf9ffee31492dbac3f649494b10dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:22.447748 16285 tablet_service.cc:1467] Processing CreateTablet for tablet 4f733400846a4489b68d717902262020 (DEFAULT_TABLE table=test-table [id=bfdaf9ffee31492dbac3f649494b10dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:22.448108 16493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f733400846a4489b68d717902262020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:22.448167 16285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f733400846a4489b68d717902262020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:22.448380 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.436889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.5.108.129:35491)
0806 01:01:22.441103 (+ 4214us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:22.441109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:22.441127 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:01:22.441199 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:22.441204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:22.441273 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:22.441373 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:22.441842 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.442572 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.444677 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.445963 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.447731 (+ 1768us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.447859 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:22.447939 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 01:01:22.448093 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4092,"thread_start_us":3973,"threads_started":1}
I20260806 01:01:22.451963 16043 tablet_service.cc:1467] Processing CreateTablet for tablet 4f733400846a4489b68d717902262020 (DEFAULT_TABLE table=test-table [id=bfdaf9ffee31492dbac3f649494b10dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:22.452486 16043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f733400846a4489b68d717902262020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:22.454843 16723 tablet_bootstrap.cc:492] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: Bootstrap starting.
I20260806 01:01:22.456030 16729 tablet_bootstrap.cc:492] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: Bootstrap starting.
I20260806 01:01:22.457352 16723 tablet_bootstrap.cc:654] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:22.457615 16729 tablet_bootstrap.cc:654] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:22.458894 16732 tablet_bootstrap.cc:492] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: Bootstrap starting.
I20260806 01:01:22.460832 16732 tablet_bootstrap.cc:654] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:22.460938 16729 tablet_bootstrap.cc:492] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: No bootstrap required, opened a new log
I20260806 01:01:22.461091 16729 ts_tablet_manager.cc:1397] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 01:01:22.462003 16729 raft_consensus.cc:348] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:22.462276 16729 raft_consensus.cc:374] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:22.462391 16729 raft_consensus.cc:729] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6e509992994b87a41b46c457c8d74b, State: Initialized, Role: FOLLOWER
I20260806 01:01:22.462664 16729 consensus_queue.cc:260] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:22.463107 16732 tablet_bootstrap.cc:492] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: No bootstrap required, opened a new log
I20260806 01:01:22.463263 16732 ts_tablet_manager.cc:1397] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 01:01:22.463551 16729 ts_tablet_manager.cc:1428] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 01:01:22.464161 16732 raft_consensus.cc:348] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:22.464437 16732 raft_consensus.cc:374] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:22.464545 16732 raft_consensus.cc:729] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e659777d823d41cc976196dc95dd78c3, State: Initialized, Role: FOLLOWER
I20260806 01:01:22.464810 16732 consensus_queue.cc:260] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:22.464996 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.463795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.131:57529)
0806 01:01:22.464388 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.464737 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":320,"threads_started":1}
I20260806 01:01:22.467219 16723 tablet_bootstrap.cc:492] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: No bootstrap required, opened a new log
I20260806 01:01:22.467408 16723 ts_tablet_manager.cc:1397] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260806 01:01:22.467420 16584 heartbeater.cc:503] Master 127.5.108.188:34699 was elected leader, sending a full tablet report...
I20260806 01:01:22.468149 16723 raft_consensus.cc:348] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:22.468423 16723 raft_consensus.cc:374] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:22.468525 16723 raft_consensus.cc:729] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad0aac9bfef4ac5883682d342760f17, State: Initialized, Role: FOLLOWER
I20260806 01:01:22.468778 16723 consensus_queue.cc:260] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:22.469695 16723 ts_tablet_manager.cc:1428] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 01:01:22.472260 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.471894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.131:33405)
0806 01:01:22.472037 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.472117 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 01:01:22.473174 16383 heartbeater.cc:503] Master 127.5.108.188:34699 was elected leader, sending a full tablet report...
I20260806 01:01:22.473199 16732 ts_tablet_manager.cc:1428] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: Time spent starting tablet: real 0.010s user 0.002s sys 0.008s
I20260806 01:01:22.475081 16741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.470447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.130:42509)
0806 01:01:22.473492 (+ 3045us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.473660 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2909,"thread_start_us":2686,"threads_started":1}
I20260806 01:01:22.475912 16156 heartbeater.cc:503] Master 127.5.108.188:34699 was elected leader, sending a full tablet report...
I20260806 01:01:22.477056 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.476084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.130:51005)
0806 01:01:22.476619 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.476783 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":278,"threads_started":1}
I20260806 01:01:22.480338 16743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.474476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.129:42921)
0806 01:01:22.480038 (+ 5562us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.480150 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5447,"thread_start_us":5136,"threads_started":1}
I20260806 01:01:22.481134 16743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.480825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.129:42463)
0806 01:01:22.480976 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.481041 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 01:01:22.776722 16736 raft_consensus.cc:482] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:22.776973 16736 raft_consensus.cc:504] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:22.778647 16736 leader_election.cc:288] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e659777d823d41cc976196dc95dd78c3 (127.5.108.129:35491), aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927)
I20260806 01:01:22.784466 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.778980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:45373 (local address 127.5.108.129:35491)
0806 01:01:22.779253 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:22.779260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:22.779391 (+ 131us) server_negotiation.cc:407] Connection header received
0806 01:01:22.779662 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:22.779666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:22.779736 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:22.779833 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:22.780279 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.780957 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.782688 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.783549 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.783714 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.783787 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:22.784052 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 01:01:22.784180 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260806 01:01:22.784518 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.778854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35491 (local address 127.5.108.131:45373)
0806 01:01:22.779250 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.779273 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:22.779507 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:22.779857 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:22.779862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:22.779886 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:22.780124 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.780134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.781105 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.781111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.782524 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.782536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.783725 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.783731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.783903 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.783927 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:01:22.784032 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 01:01:22.784177 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":1}
I20260806 01:01:22.784775 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.779077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:34441 (local address 127.5.108.130:44927)
0806 01:01:22.779314 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:22.779322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:22.779737 (+ 415us) server_negotiation.cc:407] Connection header received
0806 01:01:22.779940 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:22.779944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:22.780001 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:22.780090 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:22.780589 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.781264 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.783011 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.783903 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.784059 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.784137 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:22.784420 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 01:01:22.784542 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 01:01:22.785014 16745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.778910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44927 (local address 127.5.108.131:34441)
0806 01:01:22.779636 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.779650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:22.779806 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:22.780117 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:22.780121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:22.780145 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:22.780445 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.780453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.781389 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.781397 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.782870 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:22.782881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.784078 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.784086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:22.784244 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.784267 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 01:01:22.784394 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 01:01:22.784522 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":458,"threads_started":1}
I20260806 01:01:22.785434 16075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f733400846a4489b68d717902262020" candidate_uuid: "7d6e509992994b87a41b46c457c8d74b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e659777d823d41cc976196dc95dd78c3" is_pre_election: true
I20260806 01:01:22.785686 16315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f733400846a4489b68d717902262020" candidate_uuid: "7d6e509992994b87a41b46c457c8d74b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad0aac9bfef4ac5883682d342760f17" is_pre_election: true
I20260806 01:01:22.785933 16315 raft_consensus.cc:2454] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6e509992994b87a41b46c457c8d74b in term 0.
I20260806 01:01:22.785910 16075 raft_consensus.cc:2454] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6e509992994b87a41b46c457c8d74b in term 0.
I20260806 01:01:22.786530 16428 leader_election.cc:302] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6e509992994b87a41b46c457c8d74b, aad0aac9bfef4ac5883682d342760f17; no voters:
I20260806 01:01:22.786960 16736 raft_consensus.cc:2781] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:01:22.787093 16736 raft_consensus.cc:482] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:01:22.787199 16736 raft_consensus.cc:3037] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:22.789368 16736 raft_consensus.cc:504] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:22.789884 16736 leader_election.cc:288] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [CANDIDATE]: Term 1 election: Requested vote from peers e659777d823d41cc976196dc95dd78c3 (127.5.108.129:35491), aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927)
I20260806 01:01:22.790297 16075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f733400846a4489b68d717902262020" candidate_uuid: "7d6e509992994b87a41b46c457c8d74b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e659777d823d41cc976196dc95dd78c3"
I20260806 01:01:22.790382 16315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f733400846a4489b68d717902262020" candidate_uuid: "7d6e509992994b87a41b46c457c8d74b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad0aac9bfef4ac5883682d342760f17"
I20260806 01:01:22.790500 16075 raft_consensus.cc:3037] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:22.790602 16315 raft_consensus.cc:3037] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:22.793596 16315 raft_consensus.cc:2454] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6e509992994b87a41b46c457c8d74b in term 1.
I20260806 01:01:22.793596 16075 raft_consensus.cc:2454] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6e509992994b87a41b46c457c8d74b in term 1.
I20260806 01:01:22.793993 16428 leader_election.cc:302] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6e509992994b87a41b46c457c8d74b, aad0aac9bfef4ac5883682d342760f17; no voters:
I20260806 01:01:22.794258 16736 raft_consensus.cc:2781] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:22.794680 16736 raft_consensus.cc:686] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 1 LEADER]: Becoming Leader. State: Replica: 7d6e509992994b87a41b46c457c8d74b, State: Running, Role: LEADER
I20260806 01:01:22.795236 16736 consensus_queue.cc:237] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } }
I20260806 01:01:22.795641 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.795210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.131:41325)
0806 01:01:22.795398 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.795521 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 01:01:22.801652 15862 catalog_manager.cc:5495] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d6e509992994b87a41b46c457c8d74b (127.5.108.131). New cstate: current_term: 1 leader_uuid: "7d6e509992994b87a41b46c457c8d74b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d6e509992994b87a41b46c457c8d74b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 45995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 } health_report { overall_health: UNKNOWN } } }
I20260806 01:01:22.861660 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.861209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:41008)
0806 01:01:22.861427 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.861527 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 01:01:22.862568 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.862249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:41018)
0806 01:01:22.862366 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.862435 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 01:01:22.897217 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.891554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45178 (local address 127.5.108.131:45995)
0806 01:01:22.891772 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:22.891778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:22.891821 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:01:22.891969 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:22.891978 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:22.892054 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:22.892233 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:22.892737 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.893480 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.894816 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.895275 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.895464 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.895996 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:22.896523 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:22.896722 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:22.896787 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 01:01:22.896873 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:01:22.901451 16315 raft_consensus.cc:1261] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [term 1 FOLLOWER]: Refusing update from remote peer 7d6e509992994b87a41b46c457c8d74b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:22.901777 16075 raft_consensus.cc:1261] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [term 1 FOLLOWER]: Refusing update from remote peer 7d6e509992994b87a41b46c457c8d74b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:22.902194 16748 consensus_queue.cc:1035] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [LEADER]: Connected to new peer: Peer: permanent_uuid: "aad0aac9bfef4ac5883682d342760f17" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:22.902609 16736 consensus_queue.cc:1035] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e659777d823d41cc976196dc95dd78c3" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:22.906066 16743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.905521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.129:40841)
0806 01:01:22.905713 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.905845 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 01:01:22.908421 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.904861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.130:54279)
0806 01:01:22.908214 (+ 3353us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.908307 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3030}
I20260806 01:01:22.912693 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.912286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.130:46593)
0806 01:01:22.912420 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.912529 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 01:01:22.912776 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.912201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.131:52089)
0806 01:01:22.912491 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.912584 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260806 01:01:22.913079 16753 mvcc.cc:205] Tried to move back new op lower bound from 7315366227557093376 to 7315366227133485056. Current Snapshot: MvccSnapshot[applied={T|T < 7315366227557093376}]
I20260806 01:01:22.918125 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.917698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.131:56849)
0806 01:01:22.917911 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.918003 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 01:01:22.918849 16743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.918405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.5.108.129:57983)
0806 01:01:22.918629 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:22.918708 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260806 01:01:22.926146 15862 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:22.926430 15862 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:22.931926 15862 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9a9b8f63bc94143a5271806507d1c15: Sending DeleteTablet for 3 replicas of tablet 4f733400846a4489b68d717902262020
I20260806 01:01:22.933136 16285 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f733400846a4489b68d717902262020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:22 UTC) from {username='oryx', principal='oryx/127.5.108.188@KRBTEST.COM'} at 127.0.0.1:33516
I20260806 01:01:22.933629 16493 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f733400846a4489b68d717902262020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:22 UTC) from {username='oryx', principal='oryx/127.5.108.188@KRBTEST.COM'} at 127.0.0.1:45164
I20260806 01:01:22.933693 16043 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f733400846a4489b68d717902262020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:22 UTC) from {username='oryx', principal='oryx/127.5.108.188@KRBTEST.COM'} at 127.0.0.1:40552
I20260806 01:01:22.934885 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 15819
I20260806 01:01:22.936575 16769 tablet_replica.cc:330] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: stopping tablet replica
I20260806 01:01:22.936894 16769 raft_consensus.cc:2229] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:01:22.937677 16769 raft_consensus.cc:2258] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:22.938446 16771 tablet_replica.cc:330] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: stopping tablet replica
I20260806 01:01:22.938755 16771 raft_consensus.cc:2229] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:22.939179 16771 raft_consensus.cc:2258] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:22.939262 16769 ts_tablet_manager.cc:1905] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:22.940567 16771 ts_tablet_manager.cc:1905] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:22.947304 16765 tablet_replica.cc:330] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: stopping tablet replica
I20260806 01:01:22.947562 16765 raft_consensus.cc:2229] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:22.947870 16765 raft_consensus.cc:2258] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:22.948388 16769 ts_tablet_manager.cc:1918] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:22.948592 16769 log.cc:1196] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/4f733400846a4489b68d717902262020
I20260806 01:01:22.949146 16769 ts_tablet_manager.cc:1939] T 4f733400846a4489b68d717902262020 P 7d6e509992994b87a41b46c457c8d74b: Deleting consensus metadata
I20260806 01:01:22.949139 16771 ts_tablet_manager.cc:1918] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:22.949266 16771 log.cc:1196] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/4f733400846a4489b68d717902262020
I20260806 01:01:22.949851 16771 ts_tablet_manager.cc:1939] T 4f733400846a4489b68d717902262020 P e659777d823d41cc976196dc95dd78c3: Deleting consensus metadata
I20260806 01:01:22.950572 15842 catalog_manager.cc:4841] TS 7d6e509992994b87a41b46c457c8d74b (127.5.108.131:45995): tablet 4f733400846a4489b68d717902262020 (table test-table [id=bfdaf9ffee31492dbac3f649494b10dd]) successfully deleted
I20260806 01:01:22.951062 15840 catalog_manager.cc:4841] TS e659777d823d41cc976196dc95dd78c3 (127.5.108.129:35491): tablet 4f733400846a4489b68d717902262020 (table test-table [id=bfdaf9ffee31492dbac3f649494b10dd]) successfully deleted
I20260806 01:01:22.951630 16765 ts_tablet_manager.cc:1905] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:22.958364 16765 ts_tablet_manager.cc:1918] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:22.958490 16765 log.cc:1196] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/4f733400846a4489b68d717902262020
I20260806 01:01:22.958957 16765 ts_tablet_manager.cc:1939] T 4f733400846a4489b68d717902262020 P aad0aac9bfef4ac5883682d342760f17: Deleting consensus metadata
I20260806 01:01:22.960120 15843 catalog_manager.cc:4841] TS aad0aac9bfef4ac5883682d342760f17 (127.5.108.130:44927): tablet 4f733400846a4489b68d717902262020 (table test-table [id=bfdaf9ffee31492dbac3f649494b10dd]) successfully deleted
I20260806 01:01:22.964659 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:22.935827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40560 (local address 127.5.108.129:35491)
0806 01:01:22.936020 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:22.936029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:22.948461 (+ 12432us) server_negotiation.cc:407] Connection header received
0806 01:01:22.948682 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:22.948687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:22.948766 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:22.948869 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:22.951316 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.952362 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.954067 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:22.954654 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:22.954753 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:22.958675 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:22.958708 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:22.958729 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:22.958771 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:22.962014 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:22.962640 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:22.962646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:22.962649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:22.962778 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:22.963167 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:22.963174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:22.963177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:22.963589 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:22.963896 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:22.964111 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 01:01:22.964302 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:01:22.965297 16148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:23.088511 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 15819
I20260806 01:01:23.121981 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 16173
I20260806 01:01:23.135046 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:23.122795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33518 (local address 127.5.108.130:44927)
0806 01:01:23.122989 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:23.123010 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:23.123070 (+ 60us) server_negotiation.cc:407] Connection header received
0806 01:01:23.123270 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:23.123274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:23.123338 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:23.123468 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:23.124800 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:23.125534 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:23.126759 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:23.127139 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:23.127281 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:23.129745 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:23.129760 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:23.129773 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:23.129804 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:23.132494 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:23.133105 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:23.133112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:23.133114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:23.133242 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:23.133576 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:23.133581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:23.133584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:23.133908 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:23.134217 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:23.134473 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 01:01:23.134729 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:01:23.135758 16376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:23.244405 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 16173
I20260806 01:01:23.272856 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 16398
I20260806 01:01:23.287290 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:23.273620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45184 (local address 127.5.108.131:45995)
0806 01:01:23.273823 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:23.273829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:23.273852 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:01:23.273930 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:23.273937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:23.274051 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:23.274168 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:23.275503 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:23.276350 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:23.277581 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:23.278062 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:23.278235 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:23.280965 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:23.280978 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:23.280995 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:23.281026 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:23.284364 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:23.285172 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:23.285179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:23.285181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:23.285391 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:23.285812 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:23.285815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:23.285817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:23.286171 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:23.286405 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:23.286762 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 01:01:23.286946 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:01:23.288093 16574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:23.413982 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 16398
I20260806 01:01:23.443761 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 15346
I20260806 01:01:23.444897 15606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:23.473554 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:23.472938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:41034)
0806 01:01:23.473331 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:23.473409 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":185,"threads_started":1}
I20260806 01:01:23.474677 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:23.474393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:46687 (local address 127.0.0.1:41050)
0806 01:01:23.474514 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:23.474575 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 01:01:23.563776 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 15346
I20260806 01:01:23.592868 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 15503
I20260806 01:01:23.593860 15908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:23.713730 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 15503
2026-08-06T01:01:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10243 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:01:23 dist-test-slave-x4qh krb5kdc[16787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:01:23 dist-test-slave-x4qh krb5kdc[16787](info): set up 2 sockets
Aug 06 01:01:23 dist-test-slave-x4qh krb5kdc[16787](info): commencing operation
krb5kdc: starting...
W20260806 01:01:24.992439 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:01:25 dist-test-slave-x4qh krb5kdc[16787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:01:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.190.keytab.
Entry for principal oryx/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.190.keytab.
I20260806 01:01:25.223156 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:32799
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.190.keytab
--principal=oryx/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:32799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:25.562161 16809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:25.562439 16809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:25.562525 16809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:25.569679 16809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:25.569774 16809 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 01:01:25.569815 16809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:25.569878 16809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:25.569927 16809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:25.570005 16809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:25.579285 16809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:32799
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.190.keytab
--principal=oryx/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:32799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:25.580410 16809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:25.581670 16809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:25.594031 16818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:25.594002 16822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:25.594087 16820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:25.594884 16809 server_base.cc:1034] running on GCE node
I20260806 01:01:25.595674 16809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:25.597059 16809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:25.598310 16809 hybrid_clock.cc:648] HybridClock initialized: now 1785978085598242 us; error 49 us; skew 500 ppm
I20260806 01:01:25.598618 16809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:25 dist-test-slave-x4qh krb5kdc[16787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978085, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:25.601760 16809 init.cc:377] Logged in from keytab as oryx/127.5.108.190@KRBTEST.COM (short username oryx)
I20260806 01:01:25.603318 16809 webserver.cc:460] Webserver started at http://127.5.108.190:39435/ using document root <none> and password file <none>
I20260806 01:01:25.603981 16809 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:25.604116 16809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:25.604475 16809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:25.607451 16809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "d39bf54286834db896b016b6e451fc12"
format_stamp: "Formatted at 2026-08-06 01:01:25 on dist-test-slave-x4qh"
server_key: "d1da031fa0dc47ae5c968e5f8932346c"
server_key_iv: "113cc6fbce7715248b96f730970a970f"
server_key_version: "encryptionkey@0"
I20260806 01:01:25.608209 16809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "d39bf54286834db896b016b6e451fc12"
format_stamp: "Formatted at 2026-08-06 01:01:25 on dist-test-slave-x4qh"
server_key: "d1da031fa0dc47ae5c968e5f8932346c"
server_key_iv: "113cc6fbce7715248b96f730970a970f"
server_key_version: "encryptionkey@0"
I20260806 01:01:25.613168 16809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 01:01:25.616472 16835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:25.617712 16809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:01:25.617894 16809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "d39bf54286834db896b016b6e451fc12"
format_stamp: "Formatted at 2026-08-06 01:01:25 on dist-test-slave-x4qh"
server_key: "d1da031fa0dc47ae5c968e5f8932346c"
server_key_iv: "113cc6fbce7715248b96f730970a970f"
server_key_version: "encryptionkey@0"
I20260806 01:01:25.618103 16809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:25.642342 16809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:25.646107 16809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:25.646394 16809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:25.667899 16809 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:32799
I20260806 01:01:25.667930 16928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:32799 every 8 connection(s)
I20260806 01:01:25.669260 16809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:25.673135 16929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:25.673753 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 16809
I20260806 01:01:25.673959 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:25.674615 5554 external_mini_cluster.cc:1428] Setting key fbf029358af66d8476bca475a3181e46
I20260806 01:01:25.684971 16929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12: Bootstrap starting.
Aug 06 01:01:25 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:25.687953 16929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:25.688944 16929 log.cc:824] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:25.691780 16929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12: No bootstrap required, opened a new log
I20260806 01:01:25.695654 16933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:25.676592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.5.108.190:32799)
0806 01:01:25.677404 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:25.677415 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:25.677452 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:01:25.679013 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:25.679050 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:25.679483 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:25.679854 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:25.681325 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:25.682565 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:25.683842 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:25.684510 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:25.684709 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:25.688708 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:25.688718 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:25.688731 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:25.688766 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:25.691861 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:25.692642 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:25.692647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:25.692652 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:25.692804 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:25.693233 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:25.693238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:25.693241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:25.693685 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:25.693915 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:25.694328 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 01:01:25.694628 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":353,"threads_started":1}
I20260806 01:01:25.700556 16929 raft_consensus.cc:348] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39bf54286834db896b016b6e451fc12" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32799 } }
I20260806 01:01:25.700903 16929 raft_consensus.cc:374] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:25.701020 16929 raft_consensus.cc:729] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d39bf54286834db896b016b6e451fc12, State: Initialized, Role: FOLLOWER
I20260806 01:01:25.701485 16929 consensus_queue.cc:260] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39bf54286834db896b016b6e451fc12" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32799 } }
I20260806 01:01:25.701719 16929 raft_consensus.cc:388] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:25.701854 16929 raft_consensus.cc:482] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:25.702020 16929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:25.704064 16929 raft_consensus.cc:504] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39bf54286834db896b016b6e451fc12" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32799 } }
I20260806 01:01:25.704665 16929 leader_election.cc:302] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d39bf54286834db896b016b6e451fc12; no voters:
I20260806 01:01:25.705225 16929 leader_election.cc:288] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:25.705353 16935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:25.706060 16935 raft_consensus.cc:686] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [term 1 LEADER]: Becoming Leader. State: Replica: d39bf54286834db896b016b6e451fc12, State: Running, Role: LEADER
I20260806 01:01:25.706635 16935 consensus_queue.cc:237] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39bf54286834db896b016b6e451fc12" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32799 } }
I20260806 01:01:25.707331 16929 sys_catalog.cc:564] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:25.709405 16936 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d39bf54286834db896b016b6e451fc12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39bf54286834db896b016b6e451fc12" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32799 } } }
I20260806 01:01:25.709913 16936 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:25.709827 16939 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d39bf54286834db896b016b6e451fc12. Latest consensus state: current_term: 1 leader_uuid: "d39bf54286834db896b016b6e451fc12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39bf54286834db896b016b6e451fc12" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 32799 } } }
I20260806 01:01:25.710091 16939 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:25.711925 16947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:25.716820 16947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:25.725504 16947 catalog_manager.cc:1347] Generated new cluster ID: f9acffd6652f47a9bf563784286cfc90
I20260806 01:01:25.725602 16947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:25.748708 16947 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:25.750089 16947 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:25.755635 16947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12: Generated new TSK 0
I20260806 01:01:25.756431 16947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab.
Entry for principal oryx/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab.
I20260806 01:01:25.816622 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:32799
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab
--principal=oryx/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:26.169710 16968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:26.169963 16968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:26.170061 16968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:26.177018 16968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:26.177120 16968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:26.177204 16968 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:01:26.177289 16968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:26.186642 16968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab
--principal=oryx/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.108.190:32799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:26.187866 16968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:26.189141 16968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:26.202147 16975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:26.202191 16974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:26.202085 16978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:26.203094 16968 server_base.cc:1034] running on GCE node
I20260806 01:01:26.203745 16968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:26.204681 16968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:26.205930 16968 hybrid_clock.cc:648] HybridClock initialized: now 1785978086205864 us; error 46 us; skew 500 ppm
I20260806 01:01:26.206207 16968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:26 dist-test-slave-x4qh krb5kdc[16787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978086, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:26.209641 16968 init.cc:377] Logged in from keytab as oryx/127.5.108.129@KRBTEST.COM (short username oryx)
I20260806 01:01:26.210958 16968 webserver.cc:460] Webserver started at http://127.5.108.129:39623/ using document root <none> and password file <none>
I20260806 01:01:26.211777 16968 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:26.211859 16968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:26.212112 16968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:26.214671 16968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "d5d2355e950847918c23000f757a7318"
format_stamp: "Formatted at 2026-08-06 01:01:26 on dist-test-slave-x4qh"
server_key: "82cfb404479588768c9e5f49bce48ecd"
server_key_iv: "c0da02d70deb1cae0d2655cfaeb94da9"
server_key_version: "encryptionkey@0"
I20260806 01:01:26.215420 16968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "d5d2355e950847918c23000f757a7318"
format_stamp: "Formatted at 2026-08-06 01:01:26 on dist-test-slave-x4qh"
server_key: "82cfb404479588768c9e5f49bce48ecd"
server_key_iv: "c0da02d70deb1cae0d2655cfaeb94da9"
server_key_version: "encryptionkey@0"
I20260806 01:01:26.220185 16968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:01:26.224186 16988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:26.225466 16968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:01:26.225668 16968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "d5d2355e950847918c23000f757a7318"
format_stamp: "Formatted at 2026-08-06 01:01:26 on dist-test-slave-x4qh"
server_key: "82cfb404479588768c9e5f49bce48ecd"
server_key_iv: "c0da02d70deb1cae0d2655cfaeb94da9"
server_key_version: "encryptionkey@0"
I20260806 01:01:26.225914 16968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:26.246769 16968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:26.250687 16968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:26.250974 16968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:26.253607 16968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:26.253716 16968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:26.253875 16968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:26.253968 16968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:01:26 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978086, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:26.277370 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:26.254750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:32799 (local address 127.5.108.129:37787)
0806 01:01:26.255621 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:26.255659 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:26.257426 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:26.258236 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:26.258243 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:26.258996 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:26.259953 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:26.259988 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.261519 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.261522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:26.263761 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:26.263777 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.264948 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.264963 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:26.265140 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:26.265815 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:26.265862 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:26.269007 (+ 3145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:26.272679 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:26.272687 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:26.272701 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:26.273261 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:26.273719 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:26.273727 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:26.273733 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:26.274063 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:26.274651 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:26.274661 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:26.275011 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 01:01:26.275385 (+ 374us) client_negotiation.cc:240] Negotiation successful
0806 01:01:26.275863 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":396,"threads_started":1}
I20260806 01:01:26.278069 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:26.254114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:37787 (local address 127.5.108.190:32799)
0806 01:01:26.255120 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:26.255129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:26.257753 (+ 2624us) server_negotiation.cc:407] Connection header received
0806 01:01:26.257828 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:26.257831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:26.257890 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:26.258003 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:26.260227 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.261309 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.263966 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.264572 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.264740 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:26.269242 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:26.269248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:26.269252 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:26.269294 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:26.272554 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:26.273400 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:26.273406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:26.273409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:26.273576 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:26.274233 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:26.274237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:26.274240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:26.274519 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:26.274677 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:26.277499 (+ 2822us) server_negotiation.cc:299] Negotiation successful
0806 01:01:26.277744 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":787,"threads_started":1}
I20260806 01:01:26.301200 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:26.285198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:45065 (local address 127.5.108.190:32799)
0806 01:01:26.285365 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:26.285371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:26.286195 (+ 824us) server_negotiation.cc:407] Connection header received
0806 01:01:26.286541 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:26.286548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:26.286637 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:26.286765 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:26.288542 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.289209 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.291677 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.292094 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.292198 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:26.294912 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:26.294917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:26.294920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:26.294947 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:26.298004 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:26.299002 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:26.299008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:26.299011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:26.299172 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:26.299760 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:26.299768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:26.299771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:26.300052 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:26.300210 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:26.300620 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 01:01:26.300843 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:01:26.301302 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:26.285231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:32799 (local address 127.5.108.129:45065)
0806 01:01:26.286054 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:26.286076 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:26.286342 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:26.286825 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:26.286833 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:26.287741 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:26.288349 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:26.288370 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.289360 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.289369 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:26.291511 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:26.291530 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.292324 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.292334 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:26.292611 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:26.293335 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:26.293371 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:26.294716 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:26.298171 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:26.298181 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:26.298183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:26.298860 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:26.299320 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:26.299326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:26.299329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:26.299610 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:26.300192 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:26.300197 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:26.300374 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 01:01:26.300513 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 01:01:26.300811 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":374,"threads_started":1}
I20260806 01:01:26.311231 16968 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:36271
I20260806 01:01:26.311254 17163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:36271 every 8 connection(s)
I20260806 01:01:26.312510 16968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:26.314344 17164 heartbeater.cc:347] Connected to a master server at 127.5.108.190:32799
I20260806 01:01:26.314606 17164 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:26.315246 17164 heartbeater.cc:511] Master 127.5.108.190:32799 requested a full tablet report, sending...
I20260806 01:01:26.317348 16863 ts_manager.cc:194] Registered new tserver with Master: d5d2355e950847918c23000f757a7318 (127.5.108.129:36271)
I20260806 01:01:26.318786 16863 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:37787
I20260806 01:01:26.322081 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 16968
I20260806 01:01:26.322279 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:01:26.322952 5554 external_mini_cluster.cc:1428] Setting key a8e59e2e6dbfa25ca6b4756396cea4e7
WARNING: no policy specified for oryx/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab.
Entry for principal oryx/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab.
I20260806 01:01:26.376816 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:32799
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab
--principal=oryx/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:26.708127 17173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:26.708477 17173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:26.708648 17173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:26.715435 17173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:26.715557 17173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:26.715692 17173 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:01:26.715804 17173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:26.724942 17173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab
--principal=oryx/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.108.190:32799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:26.726081 17173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:26.727433 17173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:26.740425 17188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:26.740367 17190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:26.740899 17186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:26.741367 17173 server_base.cc:1034] running on GCE node
I20260806 01:01:26.741906 17173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:26.742715 17173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:26.743968 17173 hybrid_clock.cc:648] HybridClock initialized: now 1785978086743900 us; error 42 us; skew 500 ppm
I20260806 01:01:26.744264 17173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:26 dist-test-slave-x4qh krb5kdc[16787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978086, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:26.747314 17173 init.cc:377] Logged in from keytab as oryx/127.5.108.130@KRBTEST.COM (short username oryx)
I20260806 01:01:26.748659 17173 webserver.cc:460] Webserver started at http://127.5.108.130:33211/ using document root <none> and password file <none>
I20260806 01:01:26.749341 17173 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:26.749476 17173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:26.749812 17173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:26.752697 17173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "1b5650d18ae24abcb0b1259136ad244e"
format_stamp: "Formatted at 2026-08-06 01:01:26 on dist-test-slave-x4qh"
server_key: "d8140f9294708392ee399a623e41f252"
server_key_iv: "09a396d67650826963286e61f0a17a87"
server_key_version: "encryptionkey@0"
I20260806 01:01:26.753482 17173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "1b5650d18ae24abcb0b1259136ad244e"
format_stamp: "Formatted at 2026-08-06 01:01:26 on dist-test-slave-x4qh"
server_key: "d8140f9294708392ee399a623e41f252"
server_key_iv: "09a396d67650826963286e61f0a17a87"
server_key_version: "encryptionkey@0"
I20260806 01:01:26.758639 17173 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:01:26.762311 17204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:26.763579 17173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:26.763767 17173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "1b5650d18ae24abcb0b1259136ad244e"
format_stamp: "Formatted at 2026-08-06 01:01:26 on dist-test-slave-x4qh"
server_key: "d8140f9294708392ee399a623e41f252"
server_key_iv: "09a396d67650826963286e61f0a17a87"
server_key_version: "encryptionkey@0"
I20260806 01:01:26.764000 17173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:26.781546 17173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:26.785707 17173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:26.786010 17173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:26.788883 17173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:26.788996 17173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:26.789145 17173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:26.789225 17173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:01:26 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978086, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:26.812392 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:26.789291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:40707 (local address 127.5.108.190:32799)
0806 01:01:26.789483 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:26.789489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:26.790933 (+ 1444us) server_negotiation.cc:407] Connection header received
0806 01:01:26.793209 (+ 2276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:26.793217 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:26.793343 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:26.793475 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:26.795928 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.797041 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.799702 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.800147 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.800287 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:26.804519 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:26.804528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:26.804534 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:26.804578 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:26.808337 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:26.809376 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:26.809384 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:26.809390 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:26.809576 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:26.810052 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:26.810058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:26.810061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:26.810479 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:26.810656 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:26.811544 (+ 888us) server_negotiation.cc:299] Negotiation successful
0806 01:01:26.811849 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 01:01:26.813006 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:26.789835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:32799 (local address 127.5.108.130:40707)
0806 01:01:26.790746 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:26.790803 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:26.792858 (+ 2055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:26.793763 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:26.793776 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:26.794589 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:26.795626 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:26.795657 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.797205 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.797210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:26.799535 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:26.799559 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.800440 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.800448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:26.800647 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:26.801311 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:26.801332 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:26.804283 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:26.808525 (+ 4242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:26.808537 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:26.808559 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:26.809231 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:26.809704 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:26.809707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:26.809709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:26.809916 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:26.810623 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:26.810634 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:26.811002 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 01:01:26.811342 (+ 340us) client_negotiation.cc:240] Negotiation successful
0806 01:01:26.811872 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":412,"threads_started":1}
I20260806 01:01:26.837790 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:26.823416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:36475 (local address 127.5.108.190:32799)
0806 01:01:26.823637 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:26.823645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:26.823668 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:01:26.823897 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:26.823905 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:26.823995 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:26.824101 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:26.825399 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.826412 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.828633 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.829268 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.829409 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:26.831704 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:26.831711 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:26.831715 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:26.831760 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:26.834863 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:26.835788 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:26.835794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:26.835797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:26.835934 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:26.836318 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:26.836323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:26.836327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:26.836657 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:26.836815 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:26.837214 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 01:01:26.837429 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:01:26.837904 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:26.822922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:32799 (local address 127.5.108.130:36475)
0806 01:01:26.823564 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:26.823588 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:26.823742 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:26.824152 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:26.824157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:26.824768 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:26.825254 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:26.825273 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.826597 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.826601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:26.828468 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:26.828493 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:26.829474 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:26.829481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:26.829662 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:26.830603 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:26.830624 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:26.831503 (+ 879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:26.834980 (+ 3477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:26.834986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:26.834990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:26.835647 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:26.836058 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:26.836063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:26.836066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:26.836202 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:26.836779 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:26.836783 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:26.836964 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 01:01:26.837102 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 01:01:26.837438 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":326,"threads_started":1}
I20260806 01:01:26.847594 17173 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:39635
I20260806 01:01:26.847622 17396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:39635 every 8 connection(s)
I20260806 01:01:26.848757 17173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:26.850831 17397 heartbeater.cc:347] Connected to a master server at 127.5.108.190:32799
I20260806 01:01:26.851035 17397 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:26.851697 17397 heartbeater.cc:511] Master 127.5.108.190:32799 requested a full tablet report, sending...
I20260806 01:01:26.852975 16863 ts_manager.cc:194] Registered new tserver with Master: 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635)
I20260806 01:01:26.853565 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 17173
I20260806 01:01:26.853753 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:01:26.853770 16863 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:40707
I20260806 01:01:26.854339 5554 external_mini_cluster.cc:1428] Setting key f23e25b8be5aa9b8c413b048146bd878
WARNING: no policy specified for oryx/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab.
Entry for principal oryx/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab.
I20260806 01:01:26.910763 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:32799
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab
--principal=oryx/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:27.244573 17407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:27.244815 17407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:27.244982 17407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:27.251870 17407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:27.251984 17407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:27.252142 17407 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:01:27.252272 17407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:01:27.261425 17407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab
--principal=oryx/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.108.190:32799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:27.262548 17407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:27.263844 17407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:27.276576 17415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:27.276551 17417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:27.276857 17420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:27.277199 17407 server_base.cc:1034] running on GCE node
I20260806 01:01:27.277859 17407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:27.278656 17407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:27.279886 17407 hybrid_clock.cc:648] HybridClock initialized: now 1785978087279838 us; error 63 us; skew 500 ppm
I20260806 01:01:27.280190 17407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:27 dist-test-slave-x4qh krb5kdc[16787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978087, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:27.283243 17407 init.cc:377] Logged in from keytab as oryx/127.5.108.131@KRBTEST.COM (short username oryx)
I20260806 01:01:27.284700 17407 webserver.cc:460] Webserver started at http://127.5.108.131:45855/ using document root <none> and password file <none>
I20260806 01:01:27.285359 17407 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:27.285511 17407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:27.285877 17407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:27.288782 17407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "1e6eeae21902435fb712baef99e52f08"
format_stamp: "Formatted at 2026-08-06 01:01:27 on dist-test-slave-x4qh"
server_key: "5865723324eac5a86921c16a3119b71f"
server_key_iv: "caf261afb0ff7cb024a234e0d3d531e7"
server_key_version: "encryptionkey@0"
I20260806 01:01:27.289516 17407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "1e6eeae21902435fb712baef99e52f08"
format_stamp: "Formatted at 2026-08-06 01:01:27 on dist-test-slave-x4qh"
server_key: "5865723324eac5a86921c16a3119b71f"
server_key_iv: "caf261afb0ff7cb024a234e0d3d531e7"
server_key_version: "encryptionkey@0"
I20260806 01:01:27.294307 17407 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 01:01:27.297629 17435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:27.298760 17407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 01:01:27.298928 17407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "1e6eeae21902435fb712baef99e52f08"
format_stamp: "Formatted at 2026-08-06 01:01:27 on dist-test-slave-x4qh"
server_key: "5865723324eac5a86921c16a3119b71f"
server_key_iv: "caf261afb0ff7cb024a234e0d3d531e7"
server_key_version: "encryptionkey@0"
I20260806 01:01:27.299136 17407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:27.316282 17407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:27.320264 17407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:27.320464 17407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:27.321682 17164 heartbeater.cc:503] Master 127.5.108.190:32799 was elected leader, sending a full tablet report...
I20260806 01:01:27.323266 17407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:27.323410 17407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:27.323577 17407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:27.323675 17407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:01:27 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978087, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:27.349905 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:27.323783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:56671 (local address 127.5.108.190:32799)
0806 01:01:27.324031 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:27.324041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:27.325378 (+ 1337us) server_negotiation.cc:407] Connection header received
0806 01:01:27.327690 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:27.327697 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:27.327798 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:27.327919 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:27.330612 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.331771 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.334759 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.335492 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.335629 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:27.341453 (+ 5824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:27.341462 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:27.341468 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:27.341519 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:27.345657 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:27.346668 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:27.346676 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:27.346683 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:27.346892 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:27.347450 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:27.347457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:27.347476 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:27.347772 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:27.347968 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:27.349096 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0806 01:01:27.349389 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:01:27.351454 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:27.324353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:32799 (local address 127.5.108.131:56671)
0806 01:01:27.325207 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:27.325260 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:27.327320 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:27.328358 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:27.328371 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:27.329299 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:27.330380 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:27.330414 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.331971 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.331979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:27.334588 (+ 2609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:27.334610 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.336013 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.336022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:27.336462 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:27.337243 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:27.337307 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:27.341186 (+ 3879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:27.345813 (+ 4627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:27.345830 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:27.345847 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:27.346513 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:27.347031 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:27.347035 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:27.347037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:27.347255 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:27.347959 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:27.347968 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:27.348469 (+ 501us) client_negotiation.cc:769] Sending connection context
0806 01:01:27.348866 (+ 397us) client_negotiation.cc:240] Negotiation successful
0806 01:01:27.349534 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":384,"threads_started":1}
I20260806 01:01:27.364274 17407 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:43887
I20260806 01:01:27.364315 17612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:43887 every 8 connection(s)
I20260806 01:01:27.365482 17407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:27.367754 17614 heartbeater.cc:347] Connected to a master server at 127.5.108.190:32799
I20260806 01:01:27.367981 17614 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:27.368816 17614 heartbeater.cc:511] Master 127.5.108.190:32799 requested a full tablet report, sending...
I20260806 01:01:27.370242 16863 ts_manager.cc:194] Registered new tserver with Master: 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887)
I20260806 01:01:27.371074 16863 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:56671
I20260806 01:01:27.372390 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:27.358148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:36333 (local address 127.5.108.190:32799)
0806 01:01:27.358313 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:27.358320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:27.359428 (+ 1108us) server_negotiation.cc:407] Connection header received
0806 01:01:27.359656 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:27.359662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:27.359741 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:27.359829 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:27.361354 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.362283 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.363823 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.364210 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.364349 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:27.366137 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:27.366144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:27.366148 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:27.366186 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:27.369811 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:27.370395 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:27.370401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:27.370404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:27.370535 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:27.370956 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:27.370961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:27.370964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:27.371266 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:27.371432 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:27.371807 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 01:01:27.372008 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:01:27.372447 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:27.358426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:32799 (local address 127.5.108.131:36333)
0806 01:01:27.359312 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:27.359335 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:27.359520 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:27.360009 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:27.360015 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:27.360636 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:27.361143 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:27.361170 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.362473 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.362491 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:27.363686 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:27.363697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.364445 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.364452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:27.364611 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:27.365180 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:27.365201 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:27.365951 (+ 750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:27.369939 (+ 3988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:27.369944 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:27.369947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:27.370284 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:27.370684 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:27.370689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:27.370692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:27.370839 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:27.371389 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:27.371394 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:27.371570 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 01:01:27.371716 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 01:01:27.371985 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":443,"threads_started":1}
I20260806 01:01:27.373581 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 17407
I20260806 01:01:27.373808 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:01:27.374346 5554 external_mini_cluster.cc:1428] Setting key 724f58190ec0ef82430beb401b339d35
I20260806 01:01:27.377424 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:01:27 dist-test-slave-x4qh krb5kdc[16787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978087, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:01:27 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:27 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:27.856525 17397 heartbeater.cc:503] Master 127.5.108.190:32799 was elected leader, sending a full tablet report...
I20260806 01:01:28.374879 17614 heartbeater.cc:503] Master 127.5.108.190:32799 was elected leader, sending a full tablet report...
W20260806 01:01:28.584761 17626 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.150s user 0.008s sys 0.011s
W20260806 01:01:28.586108 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:27.424469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33844 (local address 127.5.108.190:32799)
0806 01:01:27.424710 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:27.424717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:27.424958 (+ 241us) server_negotiation.cc:407] Connection header received
0806 01:01:27.425189 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:27.425194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:27.425261 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:27.425372 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:27.426726 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.427555 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.428785 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:27.429369 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:27.429508 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.585735 (+1156227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33844: BlockingRecv error: recv got EOF from 127.0.0.1:33844 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 01:01:28 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978087, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:28.603044 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.590051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33846 (local address 127.5.108.190:32799)
0806 01:01:28.590230 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:28.590234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:28.590651 (+ 417us) server_negotiation.cc:407] Connection header received
0806 01:01:28.590795 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:28.590799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:28.590897 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:28.590986 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:28.592162 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.592824 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.594017 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.594465 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.594549 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.598194 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:28.598198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:28.598201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:28.598230 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:28.600918 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:28.601578 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:28.601582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:28.601584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:28.601690 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:28.601977 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:28.601980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:28.601982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:28.602288 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:28.602570 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:28.602673 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 01:01:28.602825 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 01:01:28.606760 16863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:01:28.613476 16863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:01:28.635728 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.625202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43814 (local address 127.5.108.129:36271)
0806 01:01:28.627637 (+ 2435us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:28.627644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:28.627671 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:01:28.627765 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:28.627772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:28.628011 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:28.629005 (+ 994us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:28.629183 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.630202 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.632861 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.634203 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.634346 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.634557 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:28.635198 (+ 641us) server_negotiation.cc:299] Negotiation successful
0806 01:01:28.635401 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2275,"thread_start_us":2110,"threads_started":1}
I20260806 01:01:28.635787 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.625016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:36271 (local address 127.0.0.1:43814)
0806 01:01:28.625744 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:28.625791 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:28.625951 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:28.628272 (+ 2321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:28.628279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:28.628305 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:28.628807 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.628823 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.630396 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.630401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.632696 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.632710 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.634705 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.634711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.634925 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.634995 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 01:01:28.635134 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 01:01:28.635325 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":97,"thread_start_us":357,"threads_started":1}
I20260806 01:01:28.636307 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.625848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.5.108.131:43887)
0806 01:01:28.626476 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:28.626484 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:28.627026 (+ 542us) server_negotiation.cc:407] Connection header received
0806 01:01:28.627249 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:28.627260 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:28.628313 (+ 1053us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:28.628501 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:28.629283 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.630958 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.633202 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.634515 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.634665 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.634870 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:28.635897 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0806 01:01:28.636066 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":308,"threads_started":1}
I20260806 01:01:28.635808 17646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.625173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:43887 (local address 127.0.0.1:43666)
0806 01:01:28.626909 (+ 1736us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:28.626922 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:28.627070 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:28.628502 (+ 1432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:28.628509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:28.628543 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:28.629071 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.629084 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.631103 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.631109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.633035 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.633049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.635030 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.635035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.635192 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.635219 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 01:01:28.635301 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 01:01:28.635428 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1630,"mutex_wait_us":1,"thread_start_us":1523,"threads_started":1}
I20260806 01:01:28.638365 17651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.627161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:39635 (local address 127.0.0.1:49720)
0806 01:01:28.630852 (+ 3691us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:28.630868 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:28.630972 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:28.631707 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:28.631712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:28.631737 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:28.632200 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.632217 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.633657 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.633662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.635410 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.635424 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.637573 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.637583 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.637756 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.637793 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:01:28.637887 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 01:01:28.638015 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3567,"mutex_wait_us":1,"thread_start_us":3406,"threads_started":1}
I20260806 01:01:28.639017 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.627720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.5.108.130:39635)
0806 01:01:28.628377 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:28.628388 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:28.631089 (+ 2701us) server_negotiation.cc:407] Connection header received
0806 01:01:28.631195 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:28.631201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:28.631509 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:28.631697 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:28.632409 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.633466 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.635594 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.637354 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.637501 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.637691 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:28.638569 (+ 878us) server_negotiation.cc:299] Negotiation successful
0806 01:01:28.638721 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":305,"threads_started":1}
I20260806 01:01:28.639839 17068 tablet_service.cc:1467] Processing CreateTablet for tablet 99b1d7d30f824d3ea82c56a8292f94e5 (DEFAULT_TABLE table=test-table [id=676bc7db0e9342c4a38fd33153b26eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:28.639915 17506 tablet_service.cc:1467] Processing CreateTablet for tablet 99b1d7d30f824d3ea82c56a8292f94e5 (DEFAULT_TABLE table=test-table [id=676bc7db0e9342c4a38fd33153b26eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:28.641116 17506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b1d7d30f824d3ea82c56a8292f94e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:28.641192 17068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b1d7d30f824d3ea82c56a8292f94e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:28.642561 17274 tablet_service.cc:1467] Processing CreateTablet for tablet 99b1d7d30f824d3ea82c56a8292f94e5 (DEFAULT_TABLE table=test-table [id=676bc7db0e9342c4a38fd33153b26eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:28.643922 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b1d7d30f824d3ea82c56a8292f94e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:28.653823 17656 tablet_bootstrap.cc:492] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: Bootstrap starting.
I20260806 01:01:28.655584 17657 tablet_bootstrap.cc:492] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: Bootstrap starting.
I20260806 01:01:28.657627 17658 tablet_bootstrap.cc:492] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: Bootstrap starting.
I20260806 01:01:28.657655 17656 tablet_bootstrap.cc:654] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:28.659188 17657 tablet_bootstrap.cc:654] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:28.659328 17656 log.cc:824] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:28.660519 17657 log.cc:824] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:28.660692 17658 tablet_bootstrap.cc:654] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:28.661818 17658 log.cc:824] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:28.663679 17656 tablet_bootstrap.cc:492] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: No bootstrap required, opened a new log
I20260806 01:01:28.663767 17657 tablet_bootstrap.cc:492] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: No bootstrap required, opened a new log
I20260806 01:01:28.663965 17656 ts_tablet_manager.cc:1397] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260806 01:01:28.664038 17657 ts_tablet_manager.cc:1397] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 01:01:28.665809 17658 tablet_bootstrap.cc:492] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: No bootstrap required, opened a new log
I20260806 01:01:28.666054 17658 ts_tablet_manager.cc:1397] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260806 01:01:28.673174 17657 raft_consensus.cc:348] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:28.673674 17657 raft_consensus.cc:374] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:28.673834 17657 raft_consensus.cc:729] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d2355e950847918c23000f757a7318, State: Initialized, Role: FOLLOWER
I20260806 01:01:28.674597 17657 consensus_queue.cc:260] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:28.674530 17656 raft_consensus.cc:348] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:28.674966 17656 raft_consensus.cc:374] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:28.675091 17656 raft_consensus.cc:729] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eeae21902435fb712baef99e52f08, State: Initialized, Role: FOLLOWER
I20260806 01:01:28.675810 17656 consensus_queue.cc:260] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:28.676654 17658 raft_consensus.cc:348] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:28.677229 17658 raft_consensus.cc:374] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:28.677420 17658 raft_consensus.cc:729] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b5650d18ae24abcb0b1259136ad244e, State: Initialized, Role: FOLLOWER
I20260806 01:01:28.678264 17658 consensus_queue.cc:260] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:28.682642 17657 ts_tablet_manager.cc:1428] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: Time spent starting tablet: real 0.018s user 0.012s sys 0.005s
I20260806 01:01:28.692106 17656 ts_tablet_manager.cc:1428] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: Time spent starting tablet: real 0.028s user 0.014s sys 0.013s
I20260806 01:01:28.692451 17658 ts_tablet_manager.cc:1428] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: Time spent starting tablet: real 0.026s user 0.016s sys 0.011s
I20260806 01:01:28.694692 17662 raft_consensus.cc:482] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:28.694976 17662 raft_consensus.cc:504] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:28.696928 17662 leader_election.cc:288] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887), 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635)
I20260806 01:01:28.705432 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.698222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:54383 (local address 127.5.108.130:39635)
0806 01:01:28.698447 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:28.698452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:28.698666 (+ 214us) server_negotiation.cc:407] Connection header received
0806 01:01:28.698909 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:28.698912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:28.698960 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:28.699025 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:28.699586 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.700496 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.703002 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.704216 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.704355 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.704451 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:28.705057 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 01:01:28.705192 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:01:28.705432 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.697283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:38243 (local address 127.5.108.131:43887)
0806 01:01:28.697443 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:28.697448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:28.698336 (+ 888us) server_negotiation.cc:407] Connection header received
0806 01:01:28.698408 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:28.698412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:28.698485 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:28.698554 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:28.699176 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.700200 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.702791 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.704166 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.704344 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.704451 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:28.705057 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 01:01:28.705192 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 01:01:28.705781 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.697965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:39635 (local address 127.5.108.129:54383)
0806 01:01:28.698585 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:28.698601 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:28.698789 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:28.699060 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:28.699065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:28.699094 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:28.699435 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.699445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.700609 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.700614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.702759 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.702772 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.704479 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.704484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.704787 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.704919 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 01:01:28.705017 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 01:01:28.705263 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":194,"spinlock_wait_cycles":17152,"thread_start_us":295,"threads_started":1}
I20260806 01:01:28.705816 17666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.697352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:43887 (local address 127.5.108.129:38243)
0806 01:01:28.697883 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:28.697917 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:28.698055 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:28.698585 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:28.698590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:28.698622 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:28.699027 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.699041 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.700315 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.700318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.702630 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.702646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.704456 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.704468 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.704819 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.704918 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 01:01:28.705017 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 01:01:28.705263 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":158,"thread_start_us":262,"threads_started":1}
I20260806 01:01:28.706710 17538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b1d7d30f824d3ea82c56a8292f94e5" candidate_uuid: "d5d2355e950847918c23000f757a7318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eeae21902435fb712baef99e52f08" is_pre_election: true
I20260806 01:01:28.706719 17321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b1d7d30f824d3ea82c56a8292f94e5" candidate_uuid: "d5d2355e950847918c23000f757a7318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b5650d18ae24abcb0b1259136ad244e" is_pre_election: true
I20260806 01:01:28.707172 17538 raft_consensus.cc:2454] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d2355e950847918c23000f757a7318 in term 0.
I20260806 01:01:28.707182 17321 raft_consensus.cc:2454] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d2355e950847918c23000f757a7318 in term 0.
I20260806 01:01:28.707808 16992 leader_election.cc:302] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6eeae21902435fb712baef99e52f08, d5d2355e950847918c23000f757a7318; no voters:
I20260806 01:01:28.708153 17662 raft_consensus.cc:2781] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:01:28.708304 17662 raft_consensus.cc:482] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:01:28.708441 17662 raft_consensus.cc:3037] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:28.710480 17662 raft_consensus.cc:504] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:28.711037 17662 leader_election.cc:288] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [CANDIDATE]: Term 1 election: Requested vote from peers 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887), 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635)
I20260806 01:01:28.711575 17321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b1d7d30f824d3ea82c56a8292f94e5" candidate_uuid: "d5d2355e950847918c23000f757a7318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b5650d18ae24abcb0b1259136ad244e"
I20260806 01:01:28.711575 17538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b1d7d30f824d3ea82c56a8292f94e5" candidate_uuid: "d5d2355e950847918c23000f757a7318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eeae21902435fb712baef99e52f08"
I20260806 01:01:28.711796 17538 raft_consensus.cc:3037] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:28.711787 17321 raft_consensus.cc:3037] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:28.714820 17538 raft_consensus.cc:2454] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d2355e950847918c23000f757a7318 in term 1.
I20260806 01:01:28.714820 17321 raft_consensus.cc:2454] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d2355e950847918c23000f757a7318 in term 1.
I20260806 01:01:28.715220 16992 leader_election.cc:302] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6eeae21902435fb712baef99e52f08, d5d2355e950847918c23000f757a7318; no voters:
I20260806 01:01:28.715474 17662 raft_consensus.cc:2781] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:28.715859 17662 raft_consensus.cc:686] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 1 LEADER]: Becoming Leader. State: Replica: d5d2355e950847918c23000f757a7318, State: Running, Role: LEADER
I20260806 01:01:28.716365 17662 consensus_queue.cc:237] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:28.720377 16861 catalog_manager.cc:5495] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5d2355e950847918c23000f757a7318 (127.5.108.129). New cstate: current_term: 1 leader_uuid: "d5d2355e950847918c23000f757a7318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } health_report { overall_health: UNKNOWN } } }
I20260806 01:01:28.741400 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.736522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43818 (local address 127.5.108.129:36271)
0806 01:01:28.736725 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:28.736731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:28.736865 (+ 134us) server_negotiation.cc:407] Connection header received
0806 01:01:28.737063 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:28.737067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:28.737134 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:28.737265 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:28.737558 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.738218 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.739440 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.739811 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.739897 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.740324 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:28.740805 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:28.740915 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:28.741031 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 01:01:28.741090 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:01:28.749061 17321 raft_consensus.cc:1261] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Refusing update from remote peer d5d2355e950847918c23000f757a7318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:28.749097 17538 raft_consensus.cc:1261] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Refusing update from remote peer d5d2355e950847918c23000f757a7318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:28.750006 17669 consensus_queue.cc:1035] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:28.750383 17662 consensus_queue.cc:1035] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:28.761374 17674 mvcc.cc:205] Tried to move back new op lower bound from 7315366251506917376 to 7315366251385622528. Current Snapshot: MvccSnapshot[applied={T|T < 7315366251506917376}]
I20260806 01:01:28.767518 17676 mvcc.cc:205] Tried to move back new op lower bound from 7315366251506917376 to 7315366251385622528. Current Snapshot: MvccSnapshot[applied={T|T < 7315366251506917376}]
I20260806 01:01:28.776207 16862 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:28.776458 16862 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:28.780073 16862 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12: Sending DeleteTablet for 3 replicas of tablet 99b1d7d30f824d3ea82c56a8292f94e5
I20260806 01:01:28.781121 17068 tablet_service.cc:1514] Processing DeleteTablet for tablet 99b1d7d30f824d3ea82c56a8292f94e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:28 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:43814
I20260806 01:01:28.781313 17274 tablet_service.cc:1514] Processing DeleteTablet for tablet 99b1d7d30f824d3ea82c56a8292f94e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:28 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:49720
I20260806 01:01:28.781584 17506 tablet_service.cc:1514] Processing DeleteTablet for tablet 99b1d7d30f824d3ea82c56a8292f94e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:28 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:43666
I20260806 01:01:28.783845 16862 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:01:28.785567 16862 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:01:28.796348 17692 tablet_replica.cc:330] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: stopping tablet replica
I20260806 01:01:28.797204 17067 tablet_service.cc:1467] Processing CreateTablet for tablet d642c58dd45c44bd866db47c01387718 (DEFAULT_TABLE table=test-table [id=e6d68d43b4614601a42f872f3d842706]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:28.797974 17067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d642c58dd45c44bd866db47c01387718. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:28.800731 17273 tablet_service.cc:1467] Processing CreateTablet for tablet d642c58dd45c44bd866db47c01387718 (DEFAULT_TABLE table=test-table [id=e6d68d43b4614601a42f872f3d842706]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:28.801441 17273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d642c58dd45c44bd866db47c01387718. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:28.818805 17506 tablet_service.cc:1467] Processing CreateTablet for tablet d642c58dd45c44bd866db47c01387718 (DEFAULT_TABLE table=test-table [id=e6d68d43b4614601a42f872f3d842706]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:28.819783 17506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d642c58dd45c44bd866db47c01387718. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:28.832139 17658 tablet_bootstrap.cc:492] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: Bootstrap starting.
I20260806 01:01:28.833855 17658 tablet_bootstrap.cc:654] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:28.835805 17692 raft_consensus.cc:2229] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:28.836601 17692 raft_consensus.cc:2258] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:28.843150 17692 ts_tablet_manager.cc:1905] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:28.843650 17657 tablet_bootstrap.cc:492] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: Bootstrap starting.
I20260806 01:01:28.845975 17657 tablet_bootstrap.cc:654] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:28.852413 17658 tablet_bootstrap.cc:492] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: No bootstrap required, opened a new log
I20260806 01:01:28.852623 17658 ts_tablet_manager.cc:1397] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.004s
I20260806 01:01:28.853866 17658 raft_consensus.cc:348] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:28.854308 17658 raft_consensus.cc:374] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:28.854462 17658 raft_consensus.cc:729] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b5650d18ae24abcb0b1259136ad244e, State: Initialized, Role: FOLLOWER
I20260806 01:01:28.854871 17658 consensus_queue.cc:260] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:28.857092 17658 ts_tablet_manager.cc:1428] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260806 01:01:28.870663 17688 tablet_replica.cc:330] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: stopping tablet replica
I20260806 01:01:28.871825 17688 raft_consensus.cc:2229] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:01:28.873363 17688 raft_consensus.cc:2258] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:28.876977 17688 ts_tablet_manager.cc:1905] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:28.881100 17657 tablet_bootstrap.cc:492] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: No bootstrap required, opened a new log
I20260806 01:01:28.881237 17656 tablet_bootstrap.cc:492] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: Bootstrap starting.
I20260806 01:01:28.881400 17657 ts_tablet_manager.cc:1397] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: Time spent bootstrapping tablet: real 0.038s user 0.010s sys 0.004s
I20260806 01:01:28.882390 17657 raft_consensus.cc:348] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:28.882720 17657 raft_consensus.cc:374] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:28.882879 17656 tablet_bootstrap.cc:654] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:28.882869 17657 raft_consensus.cc:729] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d2355e950847918c23000f757a7318, State: Initialized, Role: FOLLOWER
I20260806 01:01:28.883299 17657 consensus_queue.cc:260] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:28.886142 17657 ts_tablet_manager.cc:1428] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 01:01:28.887964 17689 tablet_replica.cc:330] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: stopping tablet replica
I20260806 01:01:28.888212 17688 ts_tablet_manager.cc:1918] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:28.888446 17688 log.cc:1196] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/99b1d7d30f824d3ea82c56a8292f94e5
I20260806 01:01:28.888994 17689 raft_consensus.cc:2229] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:28.889081 17688 ts_tablet_manager.cc:1939] T 99b1d7d30f824d3ea82c56a8292f94e5 P d5d2355e950847918c23000f757a7318: Deleting consensus metadata
I20260806 01:01:28.889561 17689 raft_consensus.cc:2258] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:28.890228 17656 tablet_bootstrap.cc:492] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: No bootstrap required, opened a new log
I20260806 01:01:28.890458 17656 ts_tablet_manager.cc:1397] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 01:01:28.891062 16837 catalog_manager.cc:4841] TS d5d2355e950847918c23000f757a7318 (127.5.108.129:36271): tablet 99b1d7d30f824d3ea82c56a8292f94e5 (table test-table [id=676bc7db0e9342c4a38fd33153b26eab]) successfully deleted
I20260806 01:01:28.896683 17689 ts_tablet_manager.cc:1905] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:28.897213 17692 ts_tablet_manager.cc:1918] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:28.897150 17656 raft_consensus.cc:348] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:28.897527 17656 raft_consensus.cc:374] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:28.897662 17656 raft_consensus.cc:729] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eeae21902435fb712baef99e52f08, State: Initialized, Role: FOLLOWER
I20260806 01:01:28.897991 17656 consensus_queue.cc:260] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:28.900547 17656 ts_tablet_manager.cc:1428] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260806 01:01:28.897403 17692 log.cc:1196] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/99b1d7d30f824d3ea82c56a8292f94e5
I20260806 01:01:28.903206 17692 ts_tablet_manager.cc:1939] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1e6eeae21902435fb712baef99e52f08: Deleting consensus metadata
I20260806 01:01:28.904671 17689 ts_tablet_manager.cc:1918] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:28.904836 17689 log.cc:1196] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/99b1d7d30f824d3ea82c56a8292f94e5
I20260806 01:01:28.905268 16839 catalog_manager.cc:4841] TS 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887): tablet 99b1d7d30f824d3ea82c56a8292f94e5 (table test-table [id=676bc7db0e9342c4a38fd33153b26eab]) successfully deleted
I20260806 01:01:28.905421 17689 ts_tablet_manager.cc:1939] T 99b1d7d30f824d3ea82c56a8292f94e5 P 1b5650d18ae24abcb0b1259136ad244e: Deleting consensus metadata
I20260806 01:01:28.906677 16839 catalog_manager.cc:4841] TS 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635): tablet 99b1d7d30f824d3ea82c56a8292f94e5 (table test-table [id=676bc7db0e9342c4a38fd33153b26eab]) successfully deleted
I20260806 01:01:28.967056 17664 raft_consensus.cc:482] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:28.967326 17664 raft_consensus.cc:504] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:28.969250 17664 leader_election.cc:288] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5d2355e950847918c23000f757a7318 (127.5.108.129:36271), 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635)
I20260806 01:01:28.977214 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.970075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:60973 (local address 127.5.108.130:39635)
0806 01:01:28.970237 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:28.970243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:28.971014 (+ 771us) server_negotiation.cc:407] Connection header received
0806 01:01:28.971173 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:28.971177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:28.971257 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:28.971358 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:28.972049 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.973088 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.975301 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.976170 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.976367 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.976443 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:28.976791 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 01:01:28.976933 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:01:28.977326 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.969697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:39635 (local address 127.5.108.131:60973)
0806 01:01:28.970901 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:28.970917 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:28.971043 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:28.971409 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:28.971419 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:28.971444 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:28.971831 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.971840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.973213 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.973217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.975157 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.975166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.976393 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.976400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.976603 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.976643 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 01:01:28.976750 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 01:01:28.976902 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"mutex_wait_us":4,"thread_start_us":604,"threads_started":1}
I20260806 01:01:28.977540 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.969697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:51577 (local address 127.5.108.129:36271)
0806 01:01:28.969916 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:28.969924 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:28.970551 (+ 627us) server_negotiation.cc:407] Connection header received
0806 01:01:28.970903 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:28.970907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:28.970989 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:28.971084 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:28.971950 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.972724 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.975133 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.976437 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.976609 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.976680 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:28.977047 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 01:01:28.977184 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:01:28.978286 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:28.969585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:36271 (local address 127.5.108.131:51577)
0806 01:01:28.970428 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:28.970454 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:28.970699 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:28.971190 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:28.971195 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:28.971225 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:28.971755 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.971769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.972857 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.972863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.974954 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:28.974969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:28.976632 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:28.976637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:28.976852 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:28.976883 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 01:01:28.977826 (+ 943us) client_negotiation.cc:240] Negotiation successful
0806 01:01:28.977918 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":87,"thread_start_us":1001,"threads_started":1}
I20260806 01:01:28.978160 17321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d642c58dd45c44bd866db47c01387718" candidate_uuid: "1e6eeae21902435fb712baef99e52f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b5650d18ae24abcb0b1259136ad244e" is_pre_election: true
I20260806 01:01:28.978453 17321 raft_consensus.cc:2454] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6eeae21902435fb712baef99e52f08 in term 0.
I20260806 01:01:28.978942 17437 leader_election.cc:302] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b5650d18ae24abcb0b1259136ad244e, 1e6eeae21902435fb712baef99e52f08; no voters:
I20260806 01:01:28.979394 17664 raft_consensus.cc:2781] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:01:28.979516 17664 raft_consensus.cc:482] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:01:28.979300 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d642c58dd45c44bd866db47c01387718" candidate_uuid: "1e6eeae21902435fb712baef99e52f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d2355e950847918c23000f757a7318" is_pre_election: true
I20260806 01:01:28.979606 17664 raft_consensus.cc:3037] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:28.979800 17095 raft_consensus.cc:2454] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6eeae21902435fb712baef99e52f08 in term 0.
I20260806 01:01:28.982108 17664 raft_consensus.cc:504] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:28.982784 17664 leader_election.cc:288] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [CANDIDATE]: Term 1 election: Requested vote from peers d5d2355e950847918c23000f757a7318 (127.5.108.129:36271), 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635)
I20260806 01:01:28.983096 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d642c58dd45c44bd866db47c01387718" candidate_uuid: "1e6eeae21902435fb712baef99e52f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d2355e950847918c23000f757a7318"
I20260806 01:01:28.983309 17095 raft_consensus.cc:3037] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:28.983304 17321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d642c58dd45c44bd866db47c01387718" candidate_uuid: "1e6eeae21902435fb712baef99e52f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b5650d18ae24abcb0b1259136ad244e"
I20260806 01:01:28.983592 17321 raft_consensus.cc:3037] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:28.985532 17321 raft_consensus.cc:2454] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6eeae21902435fb712baef99e52f08 in term 1.
I20260806 01:01:28.985880 17437 leader_election.cc:302] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b5650d18ae24abcb0b1259136ad244e, 1e6eeae21902435fb712baef99e52f08; no voters:
I20260806 01:01:28.986187 17664 raft_consensus.cc:2781] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:28.986683 17095 raft_consensus.cc:2454] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6eeae21902435fb712baef99e52f08 in term 1.
I20260806 01:01:28.986711 17664 raft_consensus.cc:686] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 1 LEADER]: Becoming Leader. State: Replica: 1e6eeae21902435fb712baef99e52f08, State: Running, Role: LEADER
I20260806 01:01:28.987463 17664 consensus_queue.cc:237] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:28.990964 16862 catalog_manager.cc:5495] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e6eeae21902435fb712baef99e52f08 (127.5.108.131). New cstate: current_term: 1 leader_uuid: "1e6eeae21902435fb712baef99e52f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } health_report { overall_health: HEALTHY } } }
Aug 06 01:01:29 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978085, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.190@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:29.045081 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.029838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:32799 (local address 127.0.0.1:33854)
0806 01:01:29.030432 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:29.030446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:29.030554 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:29.030849 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:29.030852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:29.031493 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:29.031746 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.031754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.033054 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.033057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.033954 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.033964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.034900 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.034907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.035015 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.035686 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:29.035703 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:29.038078 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:29.042077 (+ 3999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:29.042090 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:29.042096 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:29.042892 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:29.043367 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:29.043407 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:29.043411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:29.043576 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:29.044276 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:29.044282 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:29.044457 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 01:01:29.044586 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 01:01:29.044783 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":310,"threads_started":1}
I20260806 01:01:29.045426 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.029729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33854 (local address 127.5.108.190:32799)
0806 01:01:29.029906 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:29.029911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:29.030524 (+ 613us) server_negotiation.cc:407] Connection header received
0806 01:01:29.030648 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:29.030653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:29.030719 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:29.030808 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:29.031901 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.032924 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.034101 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.034722 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.034871 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.038274 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:29.038280 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:29.038285 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:29.038321 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:29.041896 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:29.043072 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:29.043078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:29.043081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:29.043246 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:29.043717 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:29.043725 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:29.043730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:29.044115 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:29.044314 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:29.044752 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 01:01:29.044987 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":63}
I20260806 01:01:29.050516 16862 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:33854:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 01:01:29.051517 16862 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:01:29.058534 17068 tablet_service.cc:1467] Processing CreateTablet for tablet 71f22a0c932942afa689ddbe3bca9a6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55fa8e24d5c24cdaab1b3a3f6efb5262]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:01:29.059172 17068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f22a0c932942afa689ddbe3bca9a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:29.059216 17274 tablet_service.cc:1467] Processing CreateTablet for tablet 71f22a0c932942afa689ddbe3bca9a6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55fa8e24d5c24cdaab1b3a3f6efb5262]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:01:29.059300 17506 tablet_service.cc:1467] Processing CreateTablet for tablet 71f22a0c932942afa689ddbe3bca9a6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55fa8e24d5c24cdaab1b3a3f6efb5262]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:01:29.059824 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f22a0c932942afa689ddbe3bca9a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:29.059900 17506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f22a0c932942afa689ddbe3bca9a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:29.066781 17657 tablet_bootstrap.cc:492] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: Bootstrap starting.
I20260806 01:01:29.066861 17658 tablet_bootstrap.cc:492] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: Bootstrap starting.
W20260806 01:01:29.067404 17165 tablet.cc:2369] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:01:29.068567 17657 tablet_bootstrap.cc:654] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:29.068621 17658 tablet_bootstrap.cc:654] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:29.070931 17657 tablet_bootstrap.cc:492] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: No bootstrap required, opened a new log
I20260806 01:01:29.071101 17657 ts_tablet_manager.cc:1397] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260806 01:01:29.071135 17656 tablet_bootstrap.cc:492] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: Bootstrap starting.
I20260806 01:01:29.071892 17657 raft_consensus.cc:348] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:29.072126 17657 raft_consensus.cc:374] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:29.072193 17657 raft_consensus.cc:729] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d2355e950847918c23000f757a7318, State: Initialized, Role: FOLLOWER
I20260806 01:01:29.072415 17658 tablet_bootstrap.cc:492] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: No bootstrap required, opened a new log
I20260806 01:01:29.072582 17658 ts_tablet_manager.cc:1397] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 01:01:29.072428 17657 consensus_queue.cc:260] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:29.072786 17656 tablet_bootstrap.cc:654] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:29.073129 17657 ts_tablet_manager.cc:1428] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 01:01:29.073346 17658 raft_consensus.cc:348] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:29.073582 17658 raft_consensus.cc:374] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:29.073665 17658 raft_consensus.cc:729] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b5650d18ae24abcb0b1259136ad244e, State: Initialized, Role: FOLLOWER
I20260806 01:01:29.073932 17683 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.073930 17658 consensus_queue.cc:260] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:29.074378 17683 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:01:29.074556 17665 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.074899 17665 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:01:29.075362 17658 ts_tablet_manager.cc:1428] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 01:01:29.085431 17656 tablet_bootstrap.cc:492] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: No bootstrap required, opened a new log
I20260806 01:01:29.085624 17656 ts_tablet_manager.cc:1397] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.003s
I20260806 01:01:29.086215 17656 raft_consensus.cc:348] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:29.086428 17656 raft_consensus.cc:374] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:29.086524 17656 raft_consensus.cc:729] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eeae21902435fb712baef99e52f08, State: Initialized, Role: FOLLOWER
I20260806 01:01:29.086747 17656 consensus_queue.cc:260] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:29.087150 17664 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.087397 17656 ts_tablet_manager.cc:1428] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 01:01:29.087479 17664 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 01:01:29.103116 17398 tablet.cc:2369] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:01:29.118485 17615 tablet.cc:2369] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:01:29.225499 17665 raft_consensus.cc:482] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:29.225785 17665 raft_consensus.cc:504] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:29.227902 17665 leader_election.cc:288] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5d2355e950847918c23000f757a7318 (127.5.108.129:36271), 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887)
I20260806 01:01:29.250435 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.228577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:51573 (local address 127.5.108.129:36271)
0806 01:01:29.228970 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:29.228979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:29.241052 (+ 12073us) server_negotiation.cc:407] Connection header received
0806 01:01:29.241590 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:29.241599 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:29.241709 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:29.241789 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:29.242721 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.244059 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.247353 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.248946 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.249110 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.249212 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:29.249897 (+ 685us) server_negotiation.cc:299] Negotiation successful
0806 01:01:29.250080 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:01:29.251456 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.228191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:36271 (local address 127.5.108.130:51573)
0806 01:01:29.240924 (+ 12733us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:29.240949 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:29.241391 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:29.242049 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:29.242057 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:29.242089 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:29.242498 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.242513 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.244278 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.244286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.247117 (+ 2831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.247138 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.249441 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.249449 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.249688 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.249734 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 01:01:29.250750 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0806 01:01:29.250936 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12557,"thread_start_us":12876,"threads_started":1}
I20260806 01:01:29.252355 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f22a0c932942afa689ddbe3bca9a6f" candidate_uuid: "1b5650d18ae24abcb0b1259136ad244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d2355e950847918c23000f757a7318" is_pre_election: true
I20260806 01:01:29.252683 17095 raft_consensus.cc:2454] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b5650d18ae24abcb0b1259136ad244e in term 0.
I20260806 01:01:29.253330 17206 leader_election.cc:302] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b5650d18ae24abcb0b1259136ad244e, d5d2355e950847918c23000f757a7318; no voters:
I20260806 01:01:29.253860 17665 raft_consensus.cc:2781] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:01:29.253996 17665 raft_consensus.cc:482] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:01:29.254107 17665 raft_consensus.cc:3037] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:29.257510 17665 raft_consensus.cc:504] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:29.258662 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f22a0c932942afa689ddbe3bca9a6f" candidate_uuid: "1b5650d18ae24abcb0b1259136ad244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d2355e950847918c23000f757a7318"
I20260806 01:01:29.258908 17095 raft_consensus.cc:3037] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:29.261973 17095 raft_consensus.cc:2454] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b5650d18ae24abcb0b1259136ad244e in term 1.
I20260806 01:01:29.262468 17206 leader_election.cc:302] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b5650d18ae24abcb0b1259136ad244e, d5d2355e950847918c23000f757a7318; no voters:
I20260806 01:01:29.264843 17665 leader_election.cc:288] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [CANDIDATE]: Term 1 election: Requested vote from peers d5d2355e950847918c23000f757a7318 (127.5.108.129:36271), 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887)
I20260806 01:01:29.265309 17665 raft_consensus.cc:2781] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:29.265661 17665 raft_consensus.cc:686] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [term 1 LEADER]: Becoming Leader. State: Replica: 1b5650d18ae24abcb0b1259136ad244e, State: Running, Role: LEADER
I20260806 01:01:29.266270 17665 consensus_queue.cc:237] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } }
I20260806 01:01:29.267150 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.228628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:43887 (local address 127.5.108.130:36701)
0806 01:01:29.252271 (+ 23643us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:29.252294 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:29.252408 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:29.258948 (+ 6540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:29.258955 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:29.258987 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:01:29.259192 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.259205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.261402 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.261410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.263747 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.263767 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.266051 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.266059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.266340 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.266391 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 01:01:29.266468 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 01:01:29.266668 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23516,"thread_start_us":37047,"threads_started":1}
I20260806 01:01:29.268482 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.228992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:36701 (local address 127.5.108.131:43887)
0806 01:01:29.258448 (+ 29456us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:29.258456 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:29.258478 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:01:29.258561 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:29.258568 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:29.258676 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:29.259349 (+ 673us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:01:29.259557 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.260881 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.263986 (+ 3105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.265569 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.267630 (+ 2061us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.267759 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:29.267819 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 01:01:29.268002 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29303,"thread_start_us":29780,"threads_started":1}
I20260806 01:01:29.269243 17538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f22a0c932942afa689ddbe3bca9a6f" candidate_uuid: "1b5650d18ae24abcb0b1259136ad244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eeae21902435fb712baef99e52f08" is_pre_election: true
I20260806 01:01:29.269520 17538 raft_consensus.cc:2454] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b5650d18ae24abcb0b1259136ad244e in term 0.
I20260806 01:01:29.269474 17537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f22a0c932942afa689ddbe3bca9a6f" candidate_uuid: "1b5650d18ae24abcb0b1259136ad244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eeae21902435fb712baef99e52f08"
I20260806 01:01:29.269706 17537 raft_consensus.cc:3037] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:29.273569 17537 raft_consensus.cc:2454] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b5650d18ae24abcb0b1259136ad244e in term 1.
I20260806 01:01:29.280997 17730 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: TxnStatusTablet state changed. Reason: New leader 1b5650d18ae24abcb0b1259136ad244e. Latest consensus state: current_term: 1 leader_uuid: "1b5650d18ae24abcb0b1259136ad244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.281435 17730 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:01:29.282888 16862 catalog_manager.cc:5495] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130). New cstate: current_term: 1 leader_uuid: "1b5650d18ae24abcb0b1259136ad244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } health_report { overall_health: HEALTHY } } }
I20260806 01:01:29.286338 17733 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 01:01:29.286641 17733 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 01:01:29.355571 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.346470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:39635 (local address 127.0.0.1:49736)
0806 01:01:29.346770 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:29.346796 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:29.346984 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:29.347804 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:29.347812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:29.347828 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:01:29.348071 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.348085 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.349972 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.349980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.351935 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.351954 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.353267 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.353274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.353521 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.353614 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:01:29.354913 (+ 1299us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:01:29.354937 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:01:29.355040 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 01:01:29.355209 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 01:01:29.355684 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.346527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49736 (local address 127.5.108.130:39635)
0806 01:01:29.346720 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:29.346726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:29.347022 (+ 296us) server_negotiation.cc:407] Connection header received
0806 01:01:29.347281 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:29.347290 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:29.347454 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:29.347596 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:29.348285 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.349795 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.352132 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.352975 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.353102 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.353877 (+ 775us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:29.354780 (+ 903us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:29.354938 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:29.355102 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 01:01:29.355226 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:01:29.361007 17537 raft_consensus.cc:1261] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Refusing update from remote peer 1b5650d18ae24abcb0b1259136ad244e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:29.361162 17095 raft_consensus.cc:1261] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Refusing update from remote peer 1b5650d18ae24abcb0b1259136ad244e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:29.361816 17730 consensus_queue.cc:1035] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:29.362442 17665 consensus_queue.cc:1035] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:29.379338 17683 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: TxnStatusTablet state changed. Reason: New leader 1b5650d18ae24abcb0b1259136ad244e. Latest consensus state: current_term: 1 leader_uuid: "1b5650d18ae24abcb0b1259136ad244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.379629 17683 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:01:29.389439 17683 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b5650d18ae24abcb0b1259136ad244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.389679 17683 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P d5d2355e950847918c23000f757a7318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:01:29.389386 17664 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: TxnStatusTablet state changed. Reason: New leader 1b5650d18ae24abcb0b1259136ad244e. Latest consensus state: current_term: 1 leader_uuid: "1b5650d18ae24abcb0b1259136ad244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.391575 17664 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:01:29.393348 17665 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b5650d18ae24abcb0b1259136ad244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.393620 17665 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:01:29.394501 17665 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b5650d18ae24abcb0b1259136ad244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.394739 17665 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P 1b5650d18ae24abcb0b1259136ad244e: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:01:29.406297 17664 tablet_replica.cc:439] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b5650d18ae24abcb0b1259136ad244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } }
I20260806 01:01:29.406558 17664 tablet_replica.cc:442] T 71f22a0c932942afa689ddbe3bca9a6f P 1e6eeae21902435fb712baef99e52f08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:01:29.420270 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.414207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.5.108.131:43887)
0806 01:01:29.414407 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:29.414413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:29.414814 (+ 401us) server_negotiation.cc:407] Connection header received
0806 01:01:29.415107 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:29.415111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:29.415185 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:29.415340 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:29.415832 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.416651 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.418168 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.418650 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.418780 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.419193 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:29.419758 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:29.419917 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:29.419966 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 01:01:29.420034 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:01:29.423790 17664 consensus_queue.cc:1035] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:29.431520 17698 consensus_queue.cc:1035] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:29.444316 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.437418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:49427 (local address 127.5.108.130:39635)
0806 01:01:29.437651 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:29.437657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:29.439112 (+ 1455us) server_negotiation.cc:407] Connection header received
0806 01:01:29.439315 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:29.439319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:29.439421 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:29.439511 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:29.440021 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.440899 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.442370 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.442918 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.443069 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.443599 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:29.443695 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:29.443802 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:29.443914 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 01:01:29.443990 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:01:29.444407 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.437056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:39635 (local address 127.5.108.131:49427)
0806 01:01:29.439001 (+ 1945us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:29.439020 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:29.439166 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:29.439524 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:29.439529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:29.439539 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:01:29.439841 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.439852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.441043 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.441046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.442234 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.442245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.443099 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.443106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.443270 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.443356 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:01:29.443803 (+ 447us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:01:29.443816 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 01:01:29.443894 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 01:01:29.444021 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1788,"thread_start_us":1657,"threads_started":1}
I20260806 01:01:29.485260 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.478825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:49059 (local address 127.5.108.131:43887)
0806 01:01:29.478983 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:29.478990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:29.479824 (+ 834us) server_negotiation.cc:407] Connection header received
0806 01:01:29.480058 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:29.480064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:29.480144 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:29.480243 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:29.480861 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.481611 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.483139 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.483751 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.483857 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.484508 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:29.484666 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:29.484758 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:29.484962 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 01:01:29.485047 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 01:01:29.485489 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.478945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:43887 (local address 127.5.108.130:49059)
0806 01:01:29.479703 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:29.479721 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:29.479904 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:29.480263 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:29.480268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:29.480280 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:01:29.480687 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.480700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.481735 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.481743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.482921 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:29.482932 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.484067 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.484074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:29.484262 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.484353 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:01:29.484763 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:01:29.484775 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 01:01:29.484929 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 01:01:29.485078 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":536,"threads_started":1}
I20260806 01:01:29.528973 16863 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:29.529232 16863 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:29.532419 16863 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12: Sending DeleteTablet for 3 replicas of tablet d642c58dd45c44bd866db47c01387718
I20260806 01:01:29.533118 17068 tablet_service.cc:1514] Processing DeleteTablet for tablet d642c58dd45c44bd866db47c01387718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:29 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:43814
I20260806 01:01:29.533561 17506 tablet_service.cc:1514] Processing DeleteTablet for tablet d642c58dd45c44bd866db47c01387718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:29 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:43666
I20260806 01:01:29.534602 17274 tablet_service.cc:1514] Processing DeleteTablet for tablet d642c58dd45c44bd866db47c01387718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:29 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:49720
I20260806 01:01:29.544770 17784 tablet_replica.cc:330] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: stopping tablet replica
I20260806 01:01:29.545262 17784 raft_consensus.cc:2229] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:01:29.546147 17784 raft_consensus.cc:2258] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:29.548024 17784 ts_tablet_manager.cc:1905] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:29.562065 17785 tablet_replica.cc:330] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: stopping tablet replica
I20260806 01:01:29.562367 17782 tablet_replica.cc:330] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: stopping tablet replica
I20260806 01:01:29.562644 17782 raft_consensus.cc:2229] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:29.562928 17785 raft_consensus.cc:2229] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:29.563259 17782 raft_consensus.cc:2258] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:29.563616 17785 raft_consensus.cc:2258] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:29.564837 17782 ts_tablet_manager.cc:1905] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:29.566093 17785 ts_tablet_manager.cc:1905] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:29.568096 17784 ts_tablet_manager.cc:1918] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:01:29.568252 17784 log.cc:1196] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/d642c58dd45c44bd866db47c01387718
I20260806 01:01:29.568717 17784 ts_tablet_manager.cc:1939] T d642c58dd45c44bd866db47c01387718 P 1e6eeae21902435fb712baef99e52f08: Deleting consensus metadata
I20260806 01:01:29.569955 16839 catalog_manager.cc:4841] TS 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887): tablet d642c58dd45c44bd866db47c01387718 (table test-table [id=e6d68d43b4614601a42f872f3d842706]) successfully deleted
I20260806 01:01:29.572254 17782 ts_tablet_manager.cc:1918] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:01:29.572423 17782 log.cc:1196] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/d642c58dd45c44bd866db47c01387718
I20260806 01:01:29.572932 17782 ts_tablet_manager.cc:1939] T d642c58dd45c44bd866db47c01387718 P d5d2355e950847918c23000f757a7318: Deleting consensus metadata
I20260806 01:01:29.574009 16837 catalog_manager.cc:4841] TS d5d2355e950847918c23000f757a7318 (127.5.108.129:36271): tablet d642c58dd45c44bd866db47c01387718 (table test-table [id=e6d68d43b4614601a42f872f3d842706]) successfully deleted
I20260806 01:01:29.577915 17785 ts_tablet_manager.cc:1918] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:01:29.578109 17785 log.cc:1196] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/d642c58dd45c44bd866db47c01387718
I20260806 01:01:29.578613 17785 ts_tablet_manager.cc:1939] T d642c58dd45c44bd866db47c01387718 P 1b5650d18ae24abcb0b1259136ad244e: Deleting consensus metadata
I20260806 01:01:29.579779 16839 catalog_manager.cc:4841] TS 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635): tablet d642c58dd45c44bd866db47c01387718 (table test-table [id=e6d68d43b4614601a42f872f3d842706]) successfully deleted
Aug 06 01:01:29 dist-test-slave-x4qh krb5kdc[16787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:01:29 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:29 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:01:29.644152 17797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:29.619528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33858 (local address 127.5.108.190:32799)
0806 01:01:29.621593 (+ 2065us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:29.621600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:29.621620 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:01:29.621687 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:29.621693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:29.621778 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:29.621965 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:29.624300 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.625587 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.628040 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.628815 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.628967 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.643537 (+ 14570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33858: BlockingRecv error: recv got EOF from 127.0.0.1:33858 (error 108)
Metrics: {"server-negotiator.queue_time_us":1923,"thread_start_us":1805,"threads_started":1}
Aug 06 01:01:29 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM
I20260806 01:01:29.672407 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:29.648727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.5.108.190:32799)
0806 01:01:29.648910 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:29.648915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:29.649544 (+ 629us) server_negotiation.cc:407] Connection header received
0806 01:01:29.649742 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:29.649747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:29.649850 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:29.650008 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:29.651824 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.653185 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.655543 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:29.656400 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:29.656569 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:29.662821 (+ 6252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:29.662829 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:29.662834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:29.662876 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:29.667439 (+ 4563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:29.668598 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:29.668604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:29.668608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:29.669008 (+ 400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:29.669553 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:29.669558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:29.669562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:29.669950 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:29.670213 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:29.671512 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0806 01:01:29.671773 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:01:29.678357 16863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:01:29.679293 16863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:01:29.689137 17068 tablet_service.cc:1467] Processing CreateTablet for tablet 247e5bbbdeed4675b66680d0322834a3 (DEFAULT_TABLE table=test-table [id=00a5ff3d809e4d1f8c8dc947fcec5bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:29.689106 17274 tablet_service.cc:1467] Processing CreateTablet for tablet 247e5bbbdeed4675b66680d0322834a3 (DEFAULT_TABLE table=test-table [id=00a5ff3d809e4d1f8c8dc947fcec5bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:29.689652 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 247e5bbbdeed4675b66680d0322834a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:29.689657 17068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 247e5bbbdeed4675b66680d0322834a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:29.691921 17506 tablet_service.cc:1467] Processing CreateTablet for tablet 247e5bbbdeed4675b66680d0322834a3 (DEFAULT_TABLE table=test-table [id=00a5ff3d809e4d1f8c8dc947fcec5bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:29.692335 17506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 247e5bbbdeed4675b66680d0322834a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:29.706323 17807 tablet_bootstrap.cc:492] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: Bootstrap starting.
I20260806 01:01:29.709697 17805 tablet_bootstrap.cc:492] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: Bootstrap starting.
I20260806 01:01:29.712045 17808 tablet_bootstrap.cc:492] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: Bootstrap starting.
I20260806 01:01:29.715106 17808 tablet_bootstrap.cc:654] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:29.717439 17807 tablet_bootstrap.cc:654] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:29.719276 17805 tablet_bootstrap.cc:654] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:29.722483 17808 tablet_bootstrap.cc:492] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: No bootstrap required, opened a new log
I20260806 01:01:29.722709 17808 ts_tablet_manager.cc:1397] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 01:01:29.724282 17808 raft_consensus.cc:348] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:29.724592 17808 raft_consensus.cc:374] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:29.724704 17808 raft_consensus.cc:729] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eeae21902435fb712baef99e52f08, State: Initialized, Role: FOLLOWER
I20260806 01:01:29.725081 17808 consensus_queue.cc:260] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:29.725863 17805 tablet_bootstrap.cc:492] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: No bootstrap required, opened a new log
I20260806 01:01:29.726073 17805 ts_tablet_manager.cc:1397] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.005s
I20260806 01:01:29.727216 17805 raft_consensus.cc:348] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:29.727599 17805 raft_consensus.cc:374] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:29.727811 17805 raft_consensus.cc:729] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d2355e950847918c23000f757a7318, State: Initialized, Role: FOLLOWER
I20260806 01:01:29.728186 17805 consensus_queue.cc:260] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:29.729758 17808 ts_tablet_manager.cc:1428] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260806 01:01:29.735754 17805 ts_tablet_manager.cc:1428] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260806 01:01:29.740473 17807 tablet_bootstrap.cc:492] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: No bootstrap required, opened a new log
I20260806 01:01:29.740653 17807 ts_tablet_manager.cc:1397] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: Time spent bootstrapping tablet: real 0.034s user 0.012s sys 0.008s
I20260806 01:01:29.741933 17807 raft_consensus.cc:348] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:29.742339 17807 raft_consensus.cc:374] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:29.742568 17807 raft_consensus.cc:729] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b5650d18ae24abcb0b1259136ad244e, State: Initialized, Role: FOLLOWER
I20260806 01:01:29.742944 17807 consensus_queue.cc:260] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:29.744479 17807 ts_tablet_manager.cc:1428] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 01:01:29.946527 17730 raft_consensus.cc:482] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:29.946821 17730 raft_consensus.cc:504] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:29.947549 17730 leader_election.cc:288] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5d2355e950847918c23000f757a7318 (127.5.108.129:36271), 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887)
I20260806 01:01:29.948084 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "247e5bbbdeed4675b66680d0322834a3" candidate_uuid: "1b5650d18ae24abcb0b1259136ad244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d2355e950847918c23000f757a7318" is_pre_election: true
I20260806 01:01:29.948343 17537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "247e5bbbdeed4675b66680d0322834a3" candidate_uuid: "1b5650d18ae24abcb0b1259136ad244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eeae21902435fb712baef99e52f08" is_pre_election: true
I20260806 01:01:29.948458 17095 raft_consensus.cc:2454] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b5650d18ae24abcb0b1259136ad244e in term 0.
I20260806 01:01:29.948680 17537 raft_consensus.cc:2454] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b5650d18ae24abcb0b1259136ad244e in term 0.
I20260806 01:01:29.949033 17206 leader_election.cc:302] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b5650d18ae24abcb0b1259136ad244e, d5d2355e950847918c23000f757a7318; no voters:
I20260806 01:01:29.949460 17730 raft_consensus.cc:2781] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:01:29.949631 17730 raft_consensus.cc:482] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:01:29.949817 17730 raft_consensus.cc:3037] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:29.952268 17730 raft_consensus.cc:504] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:29.952883 17730 leader_election.cc:288] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [CANDIDATE]: Term 1 election: Requested vote from peers d5d2355e950847918c23000f757a7318 (127.5.108.129:36271), 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887)
I20260806 01:01:29.953279 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "247e5bbbdeed4675b66680d0322834a3" candidate_uuid: "1b5650d18ae24abcb0b1259136ad244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d2355e950847918c23000f757a7318"
I20260806 01:01:29.953501 17095 raft_consensus.cc:3037] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:29.953440 17537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "247e5bbbdeed4675b66680d0322834a3" candidate_uuid: "1b5650d18ae24abcb0b1259136ad244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eeae21902435fb712baef99e52f08"
I20260806 01:01:29.953709 17537 raft_consensus.cc:3037] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:29.955997 17095 raft_consensus.cc:2454] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b5650d18ae24abcb0b1259136ad244e in term 1.
I20260806 01:01:29.956413 17206 leader_election.cc:302] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b5650d18ae24abcb0b1259136ad244e, d5d2355e950847918c23000f757a7318; no voters:
I20260806 01:01:29.956670 17730 raft_consensus.cc:2781] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:29.956879 17730 raft_consensus.cc:686] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 1 LEADER]: Becoming Leader. State: Replica: 1b5650d18ae24abcb0b1259136ad244e, State: Running, Role: LEADER
I20260806 01:01:29.957139 17730 consensus_queue.cc:237] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:29.957540 17537 raft_consensus.cc:2454] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b5650d18ae24abcb0b1259136ad244e in term 1.
I20260806 01:01:29.959960 16863 catalog_manager.cc:5495] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130). New cstate: current_term: 1 leader_uuid: "1b5650d18ae24abcb0b1259136ad244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } health_report { overall_health: UNKNOWN } } }
I20260806 01:01:30.011058 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:30.004796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49744 (local address 127.5.108.130:39635)
0806 01:01:30.005244 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:30.005249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:30.005265 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:01:30.005395 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:30.005399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:30.005461 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:30.005571 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:30.006071 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.006856 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.009305 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.009683 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.009776 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:30.010382 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:30.010481 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:30.010544 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:30.010729 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 01:01:30.010788 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":247,"threads_started":1}
I20260806 01:01:30.014665 17095 raft_consensus.cc:1261] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Refusing update from remote peer 1b5650d18ae24abcb0b1259136ad244e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:30.014884 17537 raft_consensus.cc:1261] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Refusing update from remote peer 1b5650d18ae24abcb0b1259136ad244e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:30.015219 17780 consensus_queue.cc:1035] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:30.016006 17780 consensus_queue.cc:1035] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:30.040293 16862 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:30.040778 16862 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:30.049038 16862 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12: Sending DeleteTablet for 3 replicas of tablet 247e5bbbdeed4675b66680d0322834a3
I20260806 01:01:30.050272 17068 tablet_service.cc:1514] Processing DeleteTablet for tablet 247e5bbbdeed4675b66680d0322834a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:30 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:43814
I20260806 01:01:30.050631 17067 tablet_service.cc:1514] Processing DeleteTablet for tablet 247e5bbbdeed4675b66680d0322834a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:30 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:43814
I20260806 01:01:30.050866 17782 tablet_replica.cc:330] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: stopping tablet replica
I20260806 01:01:30.050824 17506 tablet_service.cc:1514] Processing DeleteTablet for tablet 247e5bbbdeed4675b66680d0322834a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:30 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:43666
I20260806 01:01:30.051066 17782 raft_consensus.cc:2229] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:30.051204 17784 tablet_replica.cc:330] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: stopping tablet replica
I20260806 01:01:30.051275 17782 raft_consensus.cc:2258] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:30.052081 17274 tablet_service.cc:1514] Processing DeleteTablet for tablet 247e5bbbdeed4675b66680d0322834a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:30 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:49720
I20260806 01:01:30.052364 17785 tablet_replica.cc:330] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: stopping tablet replica
I20260806 01:01:30.052596 17785 raft_consensus.cc:2229] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:01:30.052688 17782 ts_tablet_manager.cc:1905] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:30.053257 17785 raft_consensus.cc:2258] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:30.053853 17784 raft_consensus.cc:2229] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:30.054100 17784 raft_consensus.cc:2258] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:30.056360 17784 ts_tablet_manager.cc:1905] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:30.059425 17782 ts_tablet_manager.cc:1918] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:30.059620 17782 log.cc:1196] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/247e5bbbdeed4675b66680d0322834a3
I20260806 01:01:30.060053 17782 ts_tablet_manager.cc:1939] T 247e5bbbdeed4675b66680d0322834a3 P d5d2355e950847918c23000f757a7318: Deleting consensus metadata
I20260806 01:01:30.060431 17785 ts_tablet_manager.cc:1905] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:30.061298 16837 catalog_manager.cc:4841] TS d5d2355e950847918c23000f757a7318 (127.5.108.129:36271): tablet 247e5bbbdeed4675b66680d0322834a3 (table test-table [id=00a5ff3d809e4d1f8c8dc947fcec5bf0]) successfully deleted
W20260806 01:01:30.062305 16837 catalog_manager.cc:4808] TS d5d2355e950847918c23000f757a7318 (127.5.108.129:36271): delete failed for tablet 247e5bbbdeed4675b66680d0322834a3 because the tablet was not found. No further retry: Not found: Tablet not found: 247e5bbbdeed4675b66680d0322834a3
I20260806 01:01:30.063931 16863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:01:30.064658 16863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:01:30.072783 17506 tablet_service.cc:1467] Processing CreateTablet for tablet abb5ae9893374ab5a65b2591c0d4bf3c (DEFAULT_TABLE table=test-table [id=39f415170cb446e58529af3a85501cd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:30.073256 17506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb5ae9893374ab5a65b2591c0d4bf3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:30.076483 17784 ts_tablet_manager.cc:1918] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:30.077504 17067 tablet_service.cc:1467] Processing CreateTablet for tablet abb5ae9893374ab5a65b2591c0d4bf3c (DEFAULT_TABLE table=test-table [id=39f415170cb446e58529af3a85501cd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:30.077931 17067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb5ae9893374ab5a65b2591c0d4bf3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:30.079425 17785 ts_tablet_manager.cc:1918] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:01:30.079759 17785 log.cc:1196] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/247e5bbbdeed4675b66680d0322834a3
I20260806 01:01:30.079865 17784 log.cc:1196] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/247e5bbbdeed4675b66680d0322834a3
I20260806 01:01:30.080852 17785 ts_tablet_manager.cc:1939] T 247e5bbbdeed4675b66680d0322834a3 P 1b5650d18ae24abcb0b1259136ad244e: Deleting consensus metadata
I20260806 01:01:30.080979 17784 ts_tablet_manager.cc:1939] T 247e5bbbdeed4675b66680d0322834a3 P 1e6eeae21902435fb712baef99e52f08: Deleting consensus metadata
I20260806 01:01:30.082639 17274 tablet_service.cc:1467] Processing CreateTablet for tablet abb5ae9893374ab5a65b2591c0d4bf3c (DEFAULT_TABLE table=test-table [id=39f415170cb446e58529af3a85501cd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:01:30.083062 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb5ae9893374ab5a65b2591c0d4bf3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:30.083582 16839 catalog_manager.cc:4841] TS 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887): tablet 247e5bbbdeed4675b66680d0322834a3 (table test-table [id=00a5ff3d809e4d1f8c8dc947fcec5bf0]) successfully deleted
I20260806 01:01:30.084029 16839 catalog_manager.cc:4841] TS 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635): tablet 247e5bbbdeed4675b66680d0322834a3 (table test-table [id=00a5ff3d809e4d1f8c8dc947fcec5bf0]) successfully deleted
I20260806 01:01:30.088598 17808 tablet_bootstrap.cc:492] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: Bootstrap starting.
I20260806 01:01:30.090189 17808 tablet_bootstrap.cc:654] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:30.090521 17807 tablet_bootstrap.cc:492] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: Bootstrap starting.
I20260806 01:01:30.092209 17807 tablet_bootstrap.cc:654] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:30.092919 17805 tablet_bootstrap.cc:492] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: Bootstrap starting.
I20260806 01:01:30.094502 17805 tablet_bootstrap.cc:654] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:30.099471 17808 tablet_bootstrap.cc:492] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: No bootstrap required, opened a new log
I20260806 01:01:30.099700 17808 ts_tablet_manager.cc:1397] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.005s
I20260806 01:01:30.100524 17808 raft_consensus.cc:348] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:30.101836 17808 raft_consensus.cc:374] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:30.101984 17808 raft_consensus.cc:729] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eeae21902435fb712baef99e52f08, State: Initialized, Role: FOLLOWER
I20260806 01:01:30.102347 17808 consensus_queue.cc:260] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:30.104049 17808 ts_tablet_manager.cc:1428] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 01:01:30.105535 17805 tablet_bootstrap.cc:492] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: No bootstrap required, opened a new log
I20260806 01:01:30.105726 17805 ts_tablet_manager.cc:1397] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.002s
I20260806 01:01:30.106549 17805 raft_consensus.cc:348] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:30.106802 17805 raft_consensus.cc:374] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:30.106881 17805 raft_consensus.cc:729] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d2355e950847918c23000f757a7318, State: Initialized, Role: FOLLOWER
I20260806 01:01:30.107136 17805 consensus_queue.cc:260] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:30.108165 17807 tablet_bootstrap.cc:492] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: No bootstrap required, opened a new log
I20260806 01:01:30.108335 17807 ts_tablet_manager.cc:1397] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.003s
I20260806 01:01:30.109053 17805 ts_tablet_manager.cc:1428] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 01:01:30.109089 17807 raft_consensus.cc:348] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:30.109874 17807 raft_consensus.cc:374] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:30.109954 17807 raft_consensus.cc:729] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b5650d18ae24abcb0b1259136ad244e, State: Initialized, Role: FOLLOWER
I20260806 01:01:30.110193 17807 consensus_queue.cc:260] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:30.110792 17807 ts_tablet_manager.cc:1428] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 01:01:30.266595 17683 raft_consensus.cc:482] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:01:30.266805 17683 raft_consensus.cc:504] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:30.267604 17683 leader_election.cc:288] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635), 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887)
I20260806 01:01:30.268162 17538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb5ae9893374ab5a65b2591c0d4bf3c" candidate_uuid: "d5d2355e950847918c23000f757a7318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eeae21902435fb712baef99e52f08" is_pre_election: true
I20260806 01:01:30.268181 17321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb5ae9893374ab5a65b2591c0d4bf3c" candidate_uuid: "d5d2355e950847918c23000f757a7318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b5650d18ae24abcb0b1259136ad244e" is_pre_election: true
I20260806 01:01:30.268419 17538 raft_consensus.cc:2454] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d2355e950847918c23000f757a7318 in term 0.
I20260806 01:01:30.268486 17321 raft_consensus.cc:2454] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d2355e950847918c23000f757a7318 in term 0.
I20260806 01:01:30.268899 16992 leader_election.cc:302] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6eeae21902435fb712baef99e52f08, d5d2355e950847918c23000f757a7318; no voters:
I20260806 01:01:30.269189 17683 raft_consensus.cc:2781] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:01:30.269335 17683 raft_consensus.cc:482] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:01:30.269441 17683 raft_consensus.cc:3037] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:30.271441 17683 raft_consensus.cc:504] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:30.271991 17683 leader_election.cc:288] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [CANDIDATE]: Term 1 election: Requested vote from peers 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635), 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887)
I20260806 01:01:30.272433 17321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb5ae9893374ab5a65b2591c0d4bf3c" candidate_uuid: "d5d2355e950847918c23000f757a7318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b5650d18ae24abcb0b1259136ad244e"
I20260806 01:01:30.272557 17538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb5ae9893374ab5a65b2591c0d4bf3c" candidate_uuid: "d5d2355e950847918c23000f757a7318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eeae21902435fb712baef99e52f08"
I20260806 01:01:30.272706 17321 raft_consensus.cc:3037] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:30.272732 17538 raft_consensus.cc:3037] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:30.275560 17538 raft_consensus.cc:2454] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d2355e950847918c23000f757a7318 in term 1.
I20260806 01:01:30.275727 17321 raft_consensus.cc:2454] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d2355e950847918c23000f757a7318 in term 1.
I20260806 01:01:30.276017 16992 leader_election.cc:302] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6eeae21902435fb712baef99e52f08, d5d2355e950847918c23000f757a7318; no voters:
I20260806 01:01:30.276321 17683 raft_consensus.cc:2781] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:30.276782 17683 raft_consensus.cc:686] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 1 LEADER]: Becoming Leader. State: Replica: d5d2355e950847918c23000f757a7318, State: Running, Role: LEADER
I20260806 01:01:30.277091 17683 consensus_queue.cc:237] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } }
I20260806 01:01:30.280066 16863 catalog_manager.cc:5495] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5d2355e950847918c23000f757a7318 (127.5.108.129). New cstate: current_term: 1 leader_uuid: "d5d2355e950847918c23000f757a7318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d2355e950847918c23000f757a7318" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 36271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 } health_report { overall_health: UNKNOWN } } }
I20260806 01:01:30.307701 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:30.296779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53286 (local address 127.5.108.129:36271)
0806 01:01:30.302115 (+ 5336us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:30.302120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:30.302134 (+ 14us) server_negotiation.cc:407] Connection header received
0806 01:01:30.302185 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:30.302188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:30.302253 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:30.302333 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:30.302843 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.303587 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.305571 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.306109 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.306202 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:30.306752 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:30.306899 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:30.306968 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:30.307165 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 01:01:30.307225 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5223,"thread_start_us":5094,"threads_started":1}
I20260806 01:01:30.323688 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:30.318050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:59575 (local address 127.5.108.130:39635)
0806 01:01:30.318232 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:30.318239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:30.318303 (+ 64us) server_negotiation.cc:407] Connection header received
0806 01:01:30.318599 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:30.318604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:30.318679 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:30.318788 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:30.319321 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.320216 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.321877 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.322389 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.322521 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:30.323016 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:30.323107 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:30.323197 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:30.323318 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 01:01:30.323407 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:01:30.323809 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:30.317529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:39635 (local address 127.5.108.129:59575)
0806 01:01:30.318187 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:30.318204 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:30.318418 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:30.318802 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:30.318806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:30.318817 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:01:30.319138 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:30.319146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.320371 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.320376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:30.321742 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:30.321754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.322574 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.322581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:30.322784 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:30.322851 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:01:30.323201 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:01:30.323214 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 01:01:30.323296 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 01:01:30.323454 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":736,"threads_started":1}
I20260806 01:01:30.333611 17538 raft_consensus.cc:1261] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Refusing update from remote peer d5d2355e950847918c23000f757a7318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:30.333806 17321 raft_consensus.cc:1261] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Refusing update from remote peer d5d2355e950847918c23000f757a7318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:01:30.334349 17835 consensus_queue.cc:1035] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6eeae21902435fb712baef99e52f08" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:30.334738 17683 consensus_queue.cc:1035] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b5650d18ae24abcb0b1259136ad244e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:01:30.353950 17821 mvcc.cc:205] Tried to move back new op lower bound from 7315366258001850368 to 7315366257777479680. Current Snapshot: MvccSnapshot[applied={T|T < 7315366258001850368}]
I20260806 01:01:30.378209 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:30.370700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:42197 (local address 127.5.108.129:36271)
0806 01:01:30.370973 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:30.370983 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:30.371477 (+ 494us) server_negotiation.cc:407] Connection header received
0806 01:01:30.371762 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:30.371768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:30.371852 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:30.371955 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:01:30.373062 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.373849 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.376096 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.376737 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.376838 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:30.377433 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:01:30.377569 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:01:30.377672 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:30.377842 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 01:01:30.377935 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 01:01:30.378428 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:30.370657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:36271 (local address 127.5.108.130:42197)
0806 01:01:30.371355 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:30.371395 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:30.371586 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:30.372156 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:30.372165 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:30.372177 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:01:30.372851 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:30.372863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.373975 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.373980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:30.375629 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:30.375644 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.377005 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.377011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:30.377203 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:30.377250 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:01:30.377695 (+ 445us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:01:30.377707 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 01:01:30.377789 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 01:01:30.377947 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":698,"threads_started":1}
I20260806 01:01:30.416363 16862 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:30.416559 16862 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:01:30.419371 16862 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d39bf54286834db896b016b6e451fc12: Sending DeleteTablet for 3 replicas of tablet abb5ae9893374ab5a65b2591c0d4bf3c
I20260806 01:01:30.420050 17067 tablet_service.cc:1514] Processing DeleteTablet for tablet abb5ae9893374ab5a65b2591c0d4bf3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:30 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:43814
I20260806 01:01:30.420308 17782 tablet_replica.cc:330] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: stopping tablet replica
I20260806 01:01:30.420528 17782 raft_consensus.cc:2229] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:01:30.420555 17274 tablet_service.cc:1514] Processing DeleteTablet for tablet abb5ae9893374ab5a65b2591c0d4bf3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:30 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:49720
I20260806 01:01:30.420886 17785 tablet_replica.cc:330] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: stopping tablet replica
I20260806 01:01:30.420878 17782 raft_consensus.cc:2258] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:30.421072 17785 raft_consensus.cc:2229] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:30.421288 17785 raft_consensus.cc:2258] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:30.422165 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 16968
I20260806 01:01:30.422299 17782 ts_tablet_manager.cc:1905] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:30.422313 17785 ts_tablet_manager.cc:1905] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:30.426047 17506 tablet_service.cc:1514] Processing DeleteTablet for tablet abb5ae9893374ab5a65b2591c0d4bf3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:01:30 UTC) from {username='oryx', principal='oryx/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:43666
I20260806 01:01:30.426362 17784 tablet_replica.cc:330] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: stopping tablet replica
I20260806 01:01:30.426569 17784 raft_consensus.cc:2229] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:01:30.426774 17784 raft_consensus.cc:2258] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:01:30.429033 17784 ts_tablet_manager.cc:1905] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:01:30.429418 17785 ts_tablet_manager.cc:1918] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:01:30.429409 17782 ts_tablet_manager.cc:1918] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:01:30.429555 17785 log.cc:1196] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/abb5ae9893374ab5a65b2591c0d4bf3c
I20260806 01:01:30.429592 17782 log.cc:1196] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/abb5ae9893374ab5a65b2591c0d4bf3c
I20260806 01:01:30.430080 17785 ts_tablet_manager.cc:1939] T abb5ae9893374ab5a65b2591c0d4bf3c P 1b5650d18ae24abcb0b1259136ad244e: Deleting consensus metadata
I20260806 01:01:30.430171 17782 ts_tablet_manager.cc:1939] T abb5ae9893374ab5a65b2591c0d4bf3c P d5d2355e950847918c23000f757a7318: Deleting consensus metadata
I20260806 01:01:30.431176 16839 catalog_manager.cc:4841] TS 1b5650d18ae24abcb0b1259136ad244e (127.5.108.130:39635): tablet abb5ae9893374ab5a65b2591c0d4bf3c (table test-table [id=39f415170cb446e58529af3a85501cd2]) successfully deleted
I20260806 01:01:30.431422 16837 catalog_manager.cc:4841] TS d5d2355e950847918c23000f757a7318 (127.5.108.129:36271): tablet abb5ae9893374ab5a65b2591c0d4bf3c (table test-table [id=39f415170cb446e58529af3a85501cd2]) successfully deleted
I20260806 01:01:30.435477 17784 ts_tablet_manager.cc:1918] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:01:30.435600 17784 log.cc:1196] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/abb5ae9893374ab5a65b2591c0d4bf3c
I20260806 01:01:30.436187 17784 ts_tablet_manager.cc:1939] T abb5ae9893374ab5a65b2591c0d4bf3c P 1e6eeae21902435fb712baef99e52f08: Deleting consensus metadata
I20260806 01:01:30.437183 16839 catalog_manager.cc:4841] TS 1e6eeae21902435fb712baef99e52f08 (127.5.108.131:43887): tablet abb5ae9893374ab5a65b2591c0d4bf3c (table test-table [id=39f415170cb446e58529af3a85501cd2]) successfully deleted
Aug 06 01:01:30 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.129@KRBTEST.COM
I20260806 01:01:30.446024 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:30.423777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.5.108.129:36271)
0806 01:01:30.425890 (+ 2113us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:30.425896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:30.431744 (+ 5848us) server_negotiation.cc:407] Connection header received
0806 01:01:30.431984 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:30.431989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:30.432053 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:30.432173 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:30.433280 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.433938 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.435162 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.435745 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.435867 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:30.440633 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:30.440645 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:30.440658 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:30.440693 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:30.443321 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:30.444099 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:30.444105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:30.444107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:30.444231 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:30.444662 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:30.444665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:30.444667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:30.444993 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:30.445211 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:30.445550 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 01:01:30.445739 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1962}
I20260806 01:01:30.446656 17159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:30.565135 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 16968
I20260806 01:01:30.596565 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 17173
Aug 06 01:01:30 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.130@KRBTEST.COM
I20260806 01:01:30.610636 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:30.597496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.5.108.130:39635)
0806 01:01:30.597663 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:30.597667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:30.597683 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:01:30.597762 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:30.597765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:30.597827 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:30.597903 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:30.599006 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.599700 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.600809 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.601178 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.601260 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:30.605308 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:30.605320 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:30.605335 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:30.605363 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:30.608485 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:30.609049 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:30.609056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:30.609058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:30.609176 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:30.609457 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:30.609460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:30.609461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:30.609808 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:30.610017 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:30.610246 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 01:01:30.610385 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:01:30.611338 17391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:30.753345 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 17173
I20260806 01:01:30.789850 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 17407
Aug 06 01:01:30 dist-test-slave-x4qh krb5kdc[16787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.108.131@KRBTEST.COM
I20260806 01:01:30.804672 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:30.790655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.5.108.131:43887)
0806 01:01:30.791105 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:30.791110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:30.791134 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:01:30.791190 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:30.791194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:30.791259 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:30.791359 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:30.792595 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.793279 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.794430 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:30.794827 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:30.794912 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:30.799247 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:30.799261 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:30.799278 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:30.799310 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:30.802012 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:30.802677 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:30.802685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:30.802687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:30.802819 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:30.803224 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:30.803227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:30.803229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:30.803624 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:30.803912 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:30.804123 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 01:01:30.804300 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":242,"threads_started":1}
I20260806 01:01:30.805377 17603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:30.929083 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 17407
I20260806 01:01:30.962195 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 16809
I20260806 01:01:30.963310 16924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:31.076011 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 16809
2026-08-06T01:01:31Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7356 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:01:31 dist-test-slave-x4qh krb5kdc[17863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:01:31 dist-test-slave-x4qh krb5kdc[17863](info): set up 2 sockets
Aug 06 01:01:31 dist-test-slave-x4qh krb5kdc[17863](info): commencing operation
krb5kdc: starting...
W20260806 01:01:32.340255 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:01:32 dist-test-slave-x4qh krb5kdc[17863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:01:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:01:32.570145 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:35295
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:35295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:32.886713 17881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:32.886960 17881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:32.887143 17881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:32.893559 17881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:32.893644 17881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:32.893709 17881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:32.893846 17881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:32.893929 17881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:32.903195 17881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:35295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:35295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:32.904400 17881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:32.905669 17881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:32.917394 17891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:32.917394 17892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:32.917426 17895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:32.918061 17881 server_base.cc:1034] running on GCE node
I20260806 01:01:32.918854 17881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:32.920288 17881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:32.921489 17881 hybrid_clock.cc:648] HybridClock initialized: now 1785978092921444 us; error 62 us; skew 500 ppm
I20260806 01:01:32.921833 17881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T01:01:33Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1913 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:01:33 dist-test-slave-x4qh krb5kdc[17903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:01:33 dist-test-slave-x4qh krb5kdc[17903](info): set up 2 sockets
Aug 06 01:01:33 dist-test-slave-x4qh krb5kdc[17903](info): commencing operation
krb5kdc: starting...
W20260806 01:01:34.261264 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:01:34 dist-test-slave-x4qh krb5kdc[17903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:01:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:01:34.497246 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36523
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:34.814371 17922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:34.814666 17922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:34.814834 17922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:34.821702 17922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:34.821834 17922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:34.821938 17922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:34.822024 17922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:34.822086 17922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:34.831492 17922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:34.832648 17922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:34.833892 17922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:34.845343 17935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:34.845383 17940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:34.845410 17933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:34.846045 17922 server_base.cc:1034] running on GCE node
I20260806 01:01:34.846792 17922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:34.848101 17922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:34.849337 17922 hybrid_clock.cc:648] HybridClock initialized: now 1785978094849287 us; error 38 us; skew 500 ppm
I20260806 01:01:34.849668 17922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:34 dist-test-slave-x4qh krb5kdc[17903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978094, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:34.852739 17922 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:01:34.854074 17922 webserver.cc:460] Webserver started at http://127.5.108.190:37227/ using document root <none> and password file <none>
I20260806 01:01:34.854667 17922 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:34.854772 17922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:34.855013 17922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:34.857789 17922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c"
format_stamp: "Formatted at 2026-08-06 01:01:34 on dist-test-slave-x4qh"
server_key: "9d9478539474892d17cb81571da66527"
server_key_iv: "c9ac164905ab8d0d8c54e18a0001e00e"
server_key_version: "encryptionkey@0"
I20260806 01:01:34.858496 17922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c"
format_stamp: "Formatted at 2026-08-06 01:01:34 on dist-test-slave-x4qh"
server_key: "9d9478539474892d17cb81571da66527"
server_key_iv: "c9ac164905ab8d0d8c54e18a0001e00e"
server_key_version: "encryptionkey@0"
I20260806 01:01:34.863171 17922 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 01:01:34.866448 17948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:34.867722 17922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:34.867854 17922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c"
format_stamp: "Formatted at 2026-08-06 01:01:34 on dist-test-slave-x4qh"
server_key: "9d9478539474892d17cb81571da66527"
server_key_iv: "c9ac164905ab8d0d8c54e18a0001e00e"
server_key_version: "encryptionkey@0"
I20260806 01:01:34.868006 17922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:34.888232 17922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:34.891837 17922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:34.892040 17922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:34.909940 17922 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36523
I20260806 01:01:34.909936 18020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36523 every 8 connection(s)
I20260806 01:01:34.911278 17922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:34.914736 18021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:34.920337 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 17922
I20260806 01:01:34.920552 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:34.921177 5554 external_mini_cluster.cc:1428] Setting key b7be5279be5ea3073de1ab7d378c4f0d
I20260806 01:01:34.925210 18021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: Bootstrap starting.
I20260806 01:01:34.928189 18021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:34.929226 18021 log.cc:824] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:34.931994 18021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: No bootstrap required, opened a new log
Aug 06 01:01:34 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:34.942005 18021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } }
I20260806 01:01:34.942414 18021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:34.942528 18021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb7b190971d4bf4a4e6a7c5ca27a43c, State: Initialized, Role: FOLLOWER
I20260806 01:01:34.943167 18021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } }
I20260806 01:01:34.943486 18021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:34.943614 18021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:34.943799 18021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:34.944881 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:34.923167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.5.108.190:36523)
0806 01:01:34.923966 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:34.923977 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:34.924014 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:01:34.925947 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:34.925968 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:34.926303 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:34.926718 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:34.928314 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:34.929709 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:34.931229 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:34.931895 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:34.932014 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:34.936383 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:34.936396 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:34.936413 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:34.936457 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:34.940321 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:34.941030 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:34.941039 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:34.941048 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:34.941257 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:34.941655 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:34.941660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:34.941663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:34.942292 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:34.942599 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:34.943089 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 01:01:34.943534 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":329,"threads_started":1}
I20260806 01:01:34.946431 18021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } }
I20260806 01:01:34.946906 18021 leader_election.cc:302] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eb7b190971d4bf4a4e6a7c5ca27a43c; no voters:
I20260806 01:01:34.947554 18021 leader_election.cc:288] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:34.947705 18031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:34.948366 18031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 1 LEADER]: Becoming Leader. State: Replica: 2eb7b190971d4bf4a4e6a7c5ca27a43c, State: Running, Role: LEADER
I20260806 01:01:34.949015 18031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } }
I20260806 01:01:34.949729 18021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:34.952107 18033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eb7b190971d4bf4a4e6a7c5ca27a43c. Latest consensus state: current_term: 1 leader_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } } }
I20260806 01:01:34.952558 18033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:34.952864 18035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } } }
I20260806 01:01:34.953176 18035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:34.957168 18049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:34.961375 18049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:34.976070 18049 catalog_manager.cc:1347] Generated new cluster ID: 570fe171178f4d56abfa109495499090
I20260806 01:01:34.976203 18049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:34.984894 18049 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:34.986187 18049 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:35.000464 18049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: Generated new TSK 0
I20260806 01:01:35.001240 18049 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:01:35.060093 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36523
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:35.393676 18072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:35.393972 18072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:35.394068 18072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:35.401207 18072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:35.401314 18072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:35.401435 18072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:35.410904 18072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:35.412057 18072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:35.413311 18072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:35.426467 18082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:35.426546 18081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:35.426546 18085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:35.427665 18072 server_base.cc:1034] running on GCE node
I20260806 01:01:35.428194 18072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:35.429004 18072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:35.430207 18072 hybrid_clock.cc:648] HybridClock initialized: now 1785978095430167 us; error 29 us; skew 500 ppm
I20260806 01:01:35.430526 18072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:35 dist-test-slave-x4qh krb5kdc[17903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978095, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:35.433692 18072 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:01:35.435109 18072 webserver.cc:460] Webserver started at http://127.5.108.129:39913/ using document root <none> and password file <none>
I20260806 01:01:35.435786 18072 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:35.435885 18072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:35.436127 18072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:35.438962 18072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "09d0d4bb3b4c4a9a834304e63d3d9172"
format_stamp: "Formatted at 2026-08-06 01:01:35 on dist-test-slave-x4qh"
server_key: "86f8aa80743d1329d08e371be061711c"
server_key_iv: "b216259153a6ad750a22f3e770037b7c"
server_key_version: "encryptionkey@0"
I20260806 01:01:35.439775 18072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "09d0d4bb3b4c4a9a834304e63d3d9172"
format_stamp: "Formatted at 2026-08-06 01:01:35 on dist-test-slave-x4qh"
server_key: "86f8aa80743d1329d08e371be061711c"
server_key_iv: "b216259153a6ad750a22f3e770037b7c"
server_key_version: "encryptionkey@0"
I20260806 01:01:35.444818 18072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 01:01:35.448101 18096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:35.449344 18072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:01:35.449477 18072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "09d0d4bb3b4c4a9a834304e63d3d9172"
format_stamp: "Formatted at 2026-08-06 01:01:35 on dist-test-slave-x4qh"
server_key: "86f8aa80743d1329d08e371be061711c"
server_key_iv: "b216259153a6ad750a22f3e770037b7c"
server_key_version: "encryptionkey@0"
I20260806 01:01:35.449682 18072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:35.467823 18072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:35.471894 18072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:35.472167 18072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:35.473140 18072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:35.474367 18072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:35.474438 18072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:35.474612 18072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:35.474669 18072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:35.510484 18072 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:40397
I20260806 01:01:35.510561 18263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:40397 every 8 connection(s)
I20260806 01:01:35.512149 18072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:35.514422 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 18072
I20260806 01:01:35.514645 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:01:35.515566 5554 external_mini_cluster.cc:1428] Setting key acd280aa5e173903faa41d31ca4b5b36
Aug 06 01:01:35 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978095, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:35.534610 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:35.515442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:34425 (local address 127.5.108.190:36523)
0806 01:01:35.516278 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:35.516286 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:35.517499 (+ 1213us) server_negotiation.cc:407] Connection header received
0806 01:01:35.520262 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:35.520266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:35.520337 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:35.520423 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:35.522533 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:35.523290 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:35.524978 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:35.525465 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:35.525606 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:35.529024 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:35.529029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:35.529032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:35.529063 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:35.531663 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:35.532376 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:35.532383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:35.532390 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:35.532512 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:35.532951 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:35.532956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:35.532959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:35.533211 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:35.533364 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:35.534002 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 01:01:35.534228 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":564,"threads_started":1}
I20260806 01:01:35.535571 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:35.516396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:34425)
0806 01:01:35.517306 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:35.517365 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:35.519956 (+ 2591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:35.520687 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:35.520701 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:35.521539 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:35.522321 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:35.522349 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:35.523570 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:35.523574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:35.524816 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:35.524828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:35.525702 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:35.525711 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:35.525949 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:35.526629 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:35.526654 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:35.528806 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:35.531804 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:35.531812 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:35.531826 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:35.532237 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:35.532628 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:35.532633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:35.532637 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:35.532834 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:35.533339 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:35.533346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:35.533651 (+ 305us) client_negotiation.cc:769] Sending connection context
0806 01:01:35.533910 (+ 259us) client_negotiation.cc:240] Negotiation successful
0806 01:01:35.534382 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":358,"threads_started":1}
I20260806 01:01:35.537081 18269 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36523
I20260806 01:01:35.537451 18269 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:35.538313 18269 heartbeater.cc:511] Master 127.5.108.190:36523 requested a full tablet report, sending...
I20260806 01:01:35.540585 17975 ts_manager.cc:194] Registered new tserver with Master: 09d0d4bb3b4c4a9a834304e63d3d9172 (127.5.108.129:40397)
I20260806 01:01:35.542006 17975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:34425
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:01:35.572218 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36523
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:35.898453 18278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:35.898769 18278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:35.898947 18278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:35.906284 18278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:35.906431 18278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:35.906605 18278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:35.920514 18278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:35.921782 18278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:35.923272 18278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:35.938513 18287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:35.938586 18286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:35.938608 18290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:35.939759 18278 server_base.cc:1034] running on GCE node
I20260806 01:01:35.940373 18278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:35.941262 18278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:35.942458 18278 hybrid_clock.cc:648] HybridClock initialized: now 1785978095942403 us; error 54 us; skew 500 ppm
I20260806 01:01:35.942763 18278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:35 dist-test-slave-x4qh krb5kdc[17903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978095, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:35.948697 18278 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:01:35.950172 18278 webserver.cc:460] Webserver started at http://127.5.108.130:35499/ using document root <none> and password file <none>
I20260806 01:01:35.950810 18278 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:35.950938 18278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:35.951303 18278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:35.954385 18278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "f53665e39dc044f388471a1795c93ba9"
format_stamp: "Formatted at 2026-08-06 01:01:35 on dist-test-slave-x4qh"
server_key: "695c3fc59355dcdab474b627632df583"
server_key_iv: "85703bdc2a6b21023a89e3e256aa3f0b"
server_key_version: "encryptionkey@0"
I20260806 01:01:35.955142 18278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "f53665e39dc044f388471a1795c93ba9"
format_stamp: "Formatted at 2026-08-06 01:01:35 on dist-test-slave-x4qh"
server_key: "695c3fc59355dcdab474b627632df583"
server_key_iv: "85703bdc2a6b21023a89e3e256aa3f0b"
server_key_version: "encryptionkey@0"
I20260806 01:01:35.960144 18278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 01:01:35.963524 18298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:35.964839 18278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:01:35.965019 18278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "f53665e39dc044f388471a1795c93ba9"
format_stamp: "Formatted at 2026-08-06 01:01:35 on dist-test-slave-x4qh"
server_key: "695c3fc59355dcdab474b627632df583"
server_key_iv: "85703bdc2a6b21023a89e3e256aa3f0b"
server_key_version: "encryptionkey@0"
I20260806 01:01:35.965209 18278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:35.984484 18278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:35.988265 18278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:35.988583 18278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:35.989518 18278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:35.990871 18278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:35.990952 18278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:35.991079 18278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:35.991132 18278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:36.025195 18278 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:45749
I20260806 01:01:36.025270 18473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:45749 every 8 connection(s)
I20260806 01:01:36.026471 18278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:36.034207 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 18278
I20260806 01:01:36.034385 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:01:36.034978 5554 external_mini_cluster.cc:1428] Setting key 437615efb97ff6f09e5e9c0d4907dfa9
Aug 06 01:01:36 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978095, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:36.051007 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:36.029649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:38375 (local address 127.5.108.190:36523)
0806 01:01:36.029851 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:36.029857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:36.030963 (+ 1106us) server_negotiation.cc:407] Connection header received
0806 01:01:36.033985 (+ 3022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:36.033992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:36.034084 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:36.034206 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:36.036400 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.037614 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.039745 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.040137 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.040263 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:36.044504 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:36.044510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:36.044513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:36.044543 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:36.047262 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.048266 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.048272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.048276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.048429 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.048950 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.048958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.048964 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.049358 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:36.049522 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:36.050362 (+ 840us) server_negotiation.cc:299] Negotiation successful
0806 01:01:36.050603 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:01:36.052172 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:36.029767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:38375)
0806 01:01:36.030771 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:36.030851 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:36.033635 (+ 2784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:36.034465 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:36.034478 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:36.035271 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:36.036178 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:36.036206 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.037866 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.037871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:36.039581 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:36.039595 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.040362 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.040369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:36.040581 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:36.041454 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:36.041526 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:36.044253 (+ 2727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:36.047419 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:36.047430 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:36.047446 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:36.048107 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:36.048551 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:36.048556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:36.048561 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:36.048813 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:36.049499 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:36.049506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:36.049960 (+ 454us) client_negotiation.cc:769] Sending connection context
0806 01:01:36.050240 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 01:01:36.050803 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":441,"threads_started":1}
I20260806 01:01:36.056734 18474 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36523
I20260806 01:01:36.057045 18474 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:36.057971 18474 heartbeater.cc:511] Master 127.5.108.190:36523 requested a full tablet report, sending...
I20260806 01:01:36.059835 17975 ts_manager.cc:194] Registered new tserver with Master: f53665e39dc044f388471a1795c93ba9 (127.5.108.130:45749)
I20260806 01:01:36.060989 17975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:38375
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:01:36.102743 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36523
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:36.428177 18482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:36.428469 18482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:36.428598 18482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:36.435357 18482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:36.435530 18482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:36.435712 18482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:01:36.444690 18482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:36.445775 18482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:36.446939 18482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:36.459985 18491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:36.459906 18494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:36.459954 18492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:36.460553 18482 server_base.cc:1034] running on GCE node
I20260806 01:01:36.461180 18482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:36.462045 18482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:36.463281 18482 hybrid_clock.cc:648] HybridClock initialized: now 1785978096463236 us; error 28 us; skew 500 ppm
I20260806 01:01:36.463618 18482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:36 dist-test-slave-x4qh krb5kdc[17903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978096, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:36.466885 18482 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:01:36.468484 18482 webserver.cc:460] Webserver started at http://127.5.108.131:34395/ using document root <none> and password file <none>
I20260806 01:01:36.469229 18482 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:36.469383 18482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:36.469717 18482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:36.472762 18482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "f62f6de849814d9babcad807053124eb"
format_stamp: "Formatted at 2026-08-06 01:01:36 on dist-test-slave-x4qh"
server_key: "87d1ee7314f880d48544ca133fe5d7df"
server_key_iv: "3d13489ac93c346497cac224da8180dc"
server_key_version: "encryptionkey@0"
I20260806 01:01:36.473562 18482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "f62f6de849814d9babcad807053124eb"
format_stamp: "Formatted at 2026-08-06 01:01:36 on dist-test-slave-x4qh"
server_key: "87d1ee7314f880d48544ca133fe5d7df"
server_key_iv: "3d13489ac93c346497cac224da8180dc"
server_key_version: "encryptionkey@0"
I20260806 01:01:36.478802 18482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:01:36.482343 18505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:36.483604 18482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:36.483779 18482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "f62f6de849814d9babcad807053124eb"
format_stamp: "Formatted at 2026-08-06 01:01:36 on dist-test-slave-x4qh"
server_key: "87d1ee7314f880d48544ca133fe5d7df"
server_key_iv: "3d13489ac93c346497cac224da8180dc"
server_key_version: "encryptionkey@0"
I20260806 01:01:36.483989 18482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:36.499222 18482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:36.502859 18482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:36.503154 18482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:36.504135 18482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:36.505697 18482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:36.505847 18482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:36.506013 18482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:36.506083 18482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:36.541544 18482 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:34147
I20260806 01:01:36.541577 18672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:34147 every 8 connection(s)
I20260806 01:01:36.542780 18482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:36.545176 18269 heartbeater.cc:503] Master 127.5.108.190:36523 was elected leader, sending a full tablet report...
I20260806 01:01:36.545904 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 18482
I20260806 01:01:36.546082 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:01:36.546653 5554 external_mini_cluster.cc:1428] Setting key adfbc4593ed2aafeaf6ee03915cffdf5
Aug 06 01:01:36 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978096, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:01:36.563692 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:36.545212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:47507 (local address 127.5.108.190:36523)
0806 01:01:36.545396 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:36.545401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:36.546859 (+ 1458us) server_negotiation.cc:407] Connection header received
0806 01:01:36.549504 (+ 2645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:36.549512 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:36.549586 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:36.549703 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:36.551553 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.552340 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.554313 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.554784 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.554877 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:36.558116 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:36.558121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:36.558124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:36.558154 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:36.560708 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.561551 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.561556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.561561 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.561692 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.562124 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.562133 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.562135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.562378 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:36.562531 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:36.563154 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 01:01:36.563361 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:01:36.564538 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:36.545777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:47507)
0806 01:01:36.546676 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:36.546743 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:36.549191 (+ 2448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:36.550169 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:36.550177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:36.550716 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:36.551337 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:36.551355 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.552505 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.552509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:36.554173 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:36.554187 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.554965 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.554970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:36.555162 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:36.555800 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:36.555822 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:36.557902 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:36.560846 (+ 2944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:36.560855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:36.560868 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:36.561418 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:36.561808 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:01:36.561813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:01:36.561816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:01:36.562003 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:01:36.562508 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:36.562515 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:36.562813 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 01:01:36.563054 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 01:01:36.563514 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":365,"threads_started":1}
I20260806 01:01:36.565752 18673 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36523
I20260806 01:01:36.566062 18673 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:36.566740 18673 heartbeater.cc:511] Master 127.5.108.190:36523 requested a full tablet report, sending...
I20260806 01:01:36.568089 17975 ts_manager.cc:194] Registered new tserver with Master: f62f6de849814d9babcad807053124eb (127.5.108.131:34147)
I20260806 01:01:36.568822 17975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:47507
I20260806 01:01:36.573967 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab.
Entry for principal oryx/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab.
WARNING: no policy specified for oryx/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab.
Entry for principal oryx/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab.
WARNING: no policy specified for oryx/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab.
Entry for principal oryx/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab.
I20260806 01:01:36.712033 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 18072
Aug 06 01:01:36 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:01:36.728716 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:36.713092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59086 (local address 127.5.108.129:40397)
0806 01:01:36.713750 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:36.713758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:36.713804 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:01:36.713875 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:36.713882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:36.714091 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:36.714249 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:36.715674 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.716691 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.717957 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.718542 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.718640 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:36.722899 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:36.722911 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:36.722927 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:36.722964 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:36.725716 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.726399 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.726406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.726408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.726548 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.726970 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.726979 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.726981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.727406 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:36.727734 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:36.728090 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 01:01:36.728366 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":319,"threads_started":1}
I20260806 01:01:36.729957 18257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:36.811517 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 18072
I20260806 01:01:36.839284 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 18278
Aug 06 01:01:36 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:01:36.855604 18695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:36.840316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52322 (local address 127.5.108.130:45749)
0806 01:01:36.840967 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:36.840974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:36.841000 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:01:36.841062 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:36.841069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:36.841281 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:36.841410 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:36.842549 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.843703 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.845059 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.845627 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.845734 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:36.849834 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:36.849844 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:36.849858 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:36.849908 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:36.852749 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.853458 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.853465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.853467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.853602 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.853936 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.853939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.853941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.854397 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:36.854669 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:36.854903 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 01:01:36.855100 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":343,"threads_started":1}
I20260806 01:01:36.856900 18467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:36.943703 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 18278
I20260806 01:01:36.969897 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 18482
Aug 06 01:01:36 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:01:36.985899 18697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:36.970874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.5.108.131:34147)
0806 01:01:36.971484 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:36.971492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:36.971521 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:01:36.971579 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:36.971586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:36.971803 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:36.971969 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:36.973387 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.974346 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.975751 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:36.976290 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:36.976397 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:36.980660 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:36.980671 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:36.980698 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:36.980729 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:36.983315 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.984000 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.984007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.984009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.984125 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:36.984455 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:36.984458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:36.984460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:36.984797 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:36.985033 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:36.985259 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 01:01:36.985483 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":241,"threads_started":1}
I20260806 01:01:36.987231 18662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:37.068259 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 18482
I20260806 01:01:37.095484 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 17922
I20260806 01:01:37.096705 18014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:37.195231 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 17922
I20260806 01:01:37.224418 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36523
--webserver_interface=127.5.108.190
--webserver_port=37227
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:01:37.550849 18700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:37.551081 18700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:37.551177 18700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:37.558054 18700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:37.558198 18700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:37.558318 18700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:37.558410 18700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:37.558493 18700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:37.567349 18700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=37227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:37.568569 18700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:37.569810 18700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:37.581147 18711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:37.581292 18706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:37.581287 18707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:37.582100 18700 server_base.cc:1034] running on GCE node
I20260806 01:01:37.582908 18700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:37.584187 18700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:37.585381 18700 hybrid_clock.cc:648] HybridClock initialized: now 1785978097585340 us; error 35 us; skew 500 ppm
I20260806 01:01:37.585757 18700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:37 dist-test-slave-x4qh krb5kdc[17903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978097, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:37.588991 18700 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:01:37.590350 18700 webserver.cc:460] Webserver started at http://127.5.108.190:37227/ using document root <none> and password file <none>
I20260806 01:01:37.590947 18700 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:37.591094 18700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:37.596029 18700 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:01:37.598639 18720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:37.599793 18700 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:01:37.599957 18700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c"
format_stamp: "Formatted at 2026-08-06 01:01:34 on dist-test-slave-x4qh"
server_key: "9d9478539474892d17cb81571da66527"
server_key_iv: "c9ac164905ab8d0d8c54e18a0001e00e"
server_key_version: "encryptionkey@0"
I20260806 01:01:37.600862 18700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:37.616317 18700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:37.620435 18700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:37.620714 18700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:37.638639 18700 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36523
I20260806 01:01:37.638680 18803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36523 every 8 connection(s)
I20260806 01:01:37.640151 18700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:37.646661 18804 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:01:37.649727 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 18700
I20260806 01:01:37.650403 18804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: Bootstrap starting.
I20260806 01:01:37.650558 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:40397
--local_ip_for_outbound_sockets=127.5.108.129
--tserver_master_addrs=127.5.108.190:36523
--webserver_port=39913
--webserver_interface=127.5.108.129
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.108.129
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:01:37.668992 18804 log.cc:824] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:37.677664 18804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:01:37.678193 18804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: Bootstrap complete.
I20260806 01:01:37.686875 18804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } }
I20260806 01:01:37.687765 18804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb7b190971d4bf4a4e6a7c5ca27a43c, State: Initialized, Role: FOLLOWER
I20260806 01:01:37.688441 18804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } }
I20260806 01:01:37.688684 18804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:37.688827 18804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:37.689020 18804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:01:37.692651 18804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } }
I20260806 01:01:37.693120 18804 leader_election.cc:302] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eb7b190971d4bf4a4e6a7c5ca27a43c; no voters:
I20260806 01:01:37.693737 18804 leader_election.cc:288] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:01:37.693892 18809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:01:37.695165 18809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [term 2 LEADER]: Becoming Leader. State: Replica: 2eb7b190971d4bf4a4e6a7c5ca27a43c, State: Running, Role: LEADER
I20260806 01:01:37.695768 18809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } }
I20260806 01:01:37.696019 18804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:37.698390 18811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eb7b190971d4bf4a4e6a7c5ca27a43c. Latest consensus state: current_term: 2 leader_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } } }
I20260806 01:01:37.698740 18811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:37.698429 18810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7b190971d4bf4a4e6a7c5ca27a43c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36523 } } }
I20260806 01:01:37.698877 18810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:37.699788 18819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:37.708001 18819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:37.709723 18819 catalog_manager.cc:1259] Loaded cluster ID: 570fe171178f4d56abfa109495499090
I20260806 01:01:37.709892 18819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:37.714947 18819 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:37.716802 18819 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2eb7b190971d4bf4a4e6a7c5ca27a43c: Loaded TSK: 0
I20260806 01:01:37.718665 18819 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:01:38.013720 18807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:38.013975 18807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:38.014109 18807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:38.021044 18807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:38.021137 18807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:38.021369 18807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:38.031013 18807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.129.keytab
--principal=oryx/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:40397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=39913
--tserver_master_addrs=127.5.108.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:38.032269 18807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:38.033528 18807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:38.048101 18845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:38.048046 18846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:38.048548 18849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:38.049428 18807 server_base.cc:1034] running on GCE node
I20260806 01:01:38.050117 18807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:38.051167 18807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:38.052429 18807 hybrid_clock.cc:648] HybridClock initialized: now 1785978098052334 us; error 81 us; skew 500 ppm
I20260806 01:01:38.052799 18807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:38 dist-test-slave-x4qh krb5kdc[17903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978098, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:38.061335 18807 init.cc:377] Logged in from keytab as oryx/127.5.108.129@KRBTEST.COM (short username oryx)
I20260806 01:01:38.062849 18807 webserver.cc:460] Webserver started at http://127.5.108.129:39913/ using document root <none> and password file <none>
I20260806 01:01:38.063611 18807 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:38.063747 18807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:38.069031 18807 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 01:01:38.071676 18859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:38.072950 18807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:01:38.073133 18807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "09d0d4bb3b4c4a9a834304e63d3d9172"
format_stamp: "Formatted at 2026-08-06 01:01:35 on dist-test-slave-x4qh"
server_key: "86f8aa80743d1329d08e371be061711c"
server_key_iv: "b216259153a6ad750a22f3e770037b7c"
server_key_version: "encryptionkey@0"
I20260806 01:01:38.074003 18807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:38.091323 18807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:38.096060 18807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:38.096326 18807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:38.097362 18807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:38.098645 18807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:38.098766 18807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:38.098896 18807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:38.098994 18807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:38.133004 18807 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:40397
I20260806 01:01:38.133081 19020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:40397 every 8 connection(s)
I20260806 01:01:38.134347 18807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:38.138384 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 18807
I20260806 01:01:38.139004 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:45749
--local_ip_for_outbound_sockets=127.5.108.130
--tserver_master_addrs=127.5.108.190:36523
--webserver_port=35499
--webserver_interface=127.5.108.130
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.108.130
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:01:38 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:38 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:01:38.475982 19026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:38.476286 19026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:38.476461 19026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:38.483851 19026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:38.483983 19026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:38.484163 19026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:38.493469 19026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.130.keytab
--principal=oryx/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:45749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=35499
--tserver_master_addrs=127.5.108.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:38.494722 19026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:38.496016 19026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:38.510166 19036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:38.510582 19035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:38.510998 19041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:38.512054 19026 server_base.cc:1034] running on GCE node
I20260806 01:01:38.512735 19026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:38.513697 19026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:38.514950 19026 hybrid_clock.cc:648] HybridClock initialized: now 1785978098514845 us; error 91 us; skew 500 ppm
I20260806 01:01:38.515272 19026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:38 dist-test-slave-x4qh krb5kdc[17903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978098, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:38.518926 19026 init.cc:377] Logged in from keytab as oryx/127.5.108.130@KRBTEST.COM (short username oryx)
I20260806 01:01:38.520550 19026 webserver.cc:460] Webserver started at http://127.5.108.130:35499/ using document root <none> and password file <none>
I20260806 01:01:38.521337 19026 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:38.521519 19026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:38.526978 19026 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260806 01:01:38.530016 19050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:38.531708 19026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:01:38.531890 19026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "f53665e39dc044f388471a1795c93ba9"
format_stamp: "Formatted at 2026-08-06 01:01:35 on dist-test-slave-x4qh"
server_key: "695c3fc59355dcdab474b627632df583"
server_key_iv: "85703bdc2a6b21023a89e3e256aa3f0b"
server_key_version: "encryptionkey@0"
I20260806 01:01:38.532819 19026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:38.571018 19026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:38.578008 19026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:38.578447 19026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:38.580070 19026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:38.582415 19026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:38.582605 19026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:38.582754 19026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:38.582867 19026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:38.624084 19026 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:45749
I20260806 01:01:38.624087 19204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:45749 every 8 connection(s)
I20260806 01:01:38.626675 19026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:38.631101 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 19026
I20260806 01:01:38.631938 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:34147
--local_ip_for_outbound_sockets=127.5.108.131
--tserver_master_addrs=127.5.108.190:36523
--webserver_port=34395
--webserver_interface=127.5.108.131
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.108.131
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:01:38 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:38 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:01:39.297559 19217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:39.298133 19217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:39.298377 19217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:39.317389 19217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:39.317605 19217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:39.317919 19217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
W20260806 01:01:39.324641 19025 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.177s user 0.000s sys 0.017s
W20260806 01:01:39.326614 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:38.137072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:48177 (local address 127.5.108.190:36523)
0806 01:01:38.137721 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:38.137732 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:38.138106 (+ 374us) server_negotiation.cc:407] Connection header received
0806 01:01:38.140363 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:38.140383 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:38.140788 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:38.141165 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:38.142437 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:38.144367 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:38.145806 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:38.146882 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:38.147065 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.325912 (+1178847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:48177: BlockingRecv error: recv got EOF from 127.5.108.129:48177 (error 108)
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":264,"threads_started":1}
I20260806 01:01:39.326612 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:38.137073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:48177)
0806 01:01:38.137903 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:38.137954 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:38.139816 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:38.141281 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:38.141288 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:38.141746 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:38.142242 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:38.142259 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:38.144521 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:38.144524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:38.145660 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:38.145672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:38.147080 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:38.147084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:38.147255 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:38.147985 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:38.148006 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:39.325484 (+1177478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":361,"threads_started":1}
W20260806 01:01:39.327922 19022 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:36523 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.108.190:36523: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:39.337747 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:39.328635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:58101)
0806 01:01:39.328825 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:39.328845 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:39.328961 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:39.329354 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:39.329359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:39.329725 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:39.329922 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.329930 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.330913 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.330922 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.332049 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.332066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.332871 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.332876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.333037 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.333687 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:39.333705 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:39.337461 (+ 3756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 01:01:39.337899 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:39.328901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:58101 (local address 127.5.108.190:36523)
0806 01:01:39.329068 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:39.329073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:39.329096 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:01:39.329148 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:39.329151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:39.329207 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:39.329317 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:39.330067 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.330769 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.332227 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.332679 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.332816 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.337648 (+ 4832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:58101: BlockingRecv error: recv got EOF from 127.5.108.129:58101 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:01:39.342881 19217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/krb5kdc/oryx_127.5.108.131.keytab
--principal=oryx/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:34147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=34395
--tserver_master_addrs=127.5.108.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:39.345180 19217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:39.346895 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:39.338691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:44207)
0806 01:01:39.338836 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:39.338854 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:39.338944 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:39.339419 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:39.339424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:39.339749 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:39.339893 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.339900 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.340923 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.340928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.341848 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.341858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.342622 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.342628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.342799 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.343414 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:39.343442 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:39.346618 (+ 3176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 01:01:39.347038 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:39.338926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:44207 (local address 127.5.108.190:36523)
0806 01:01:39.339109 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:39.339113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:39.339125 (+ 12us) server_negotiation.cc:407] Connection header received
0806 01:01:39.339161 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:39.339164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:39.339222 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:39.339366 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:39.340034 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.340785 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.341986 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.342373 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.342612 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.346799 (+ 4187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:44207: BlockingRecv error: recv got EOF from 127.5.108.129:44207 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 01:01:39.347410 19022 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:01:39.347616 19217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:39.366195 19226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:39.366217 19227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:39.367825 19229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:39.368224 19217 server_base.cc:1034] running on GCE node
I20260806 01:01:39.369200 19217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:39.370708 19217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:39.372057 19217 hybrid_clock.cc:648] HybridClock initialized: now 1785978099371970 us; error 50 us; skew 500 ppm
I20260806 01:01:39.372588 19217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978099, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:01:39.377969 19217 init.cc:377] Logged in from keytab as oryx/127.5.108.131@KRBTEST.COM (short username oryx)
I20260806 01:01:39.380300 19217 webserver.cc:460] Webserver started at http://127.5.108.131:34395/ using document root <none> and password file <none>
I20260806 01:01:39.381549 19217 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:39.381768 19217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:39.391173 19217 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 01:01:39.395563 19239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:39.397778 19217 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:01:39.398032 19217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "f62f6de849814d9babcad807053124eb"
format_stamp: "Formatted at 2026-08-06 01:01:36 on dist-test-slave-x4qh"
server_key: "87d1ee7314f880d48544ca133fe5d7df"
server_key_iv: "3d13489ac93c346497cac224da8180dc"
server_key_version: "encryptionkey@0"
I20260806 01:01:39.400053 19217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:39.429168 19217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:39.436518 19217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:39.436957 19217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:39.438578 19217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:39.441013 19217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:39.441174 19217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:39.441332 19217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:39.441444 19217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:39.483454 19217 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:34147
I20260806 01:01:39.483515 19404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:34147 every 8 connection(s)
I20260806 01:01:39.485991 19217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:39.494951 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 19217
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:39.529412 19410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:39.495854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.5.108.190:36523)
0806 01:01:39.511329 (+ 15475us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:39.511337 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:39.513818 (+ 2481us) server_negotiation.cc:407] Connection header received
0806 01:01:39.513880 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:39.513885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:39.513965 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:39.514066 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:39.516841 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.518087 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.519537 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.519963 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.520095 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.523094 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:39.523103 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:39.523119 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:39.523156 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:39.526144 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:39.526864 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:39.526871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:39.526878 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:39.527041 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:01:39.527455 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:01:39.527459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:01:39.527461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:01:39.527915 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:39.528255 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:39.528705 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 01:01:39.529105 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15322,"thread_start_us":15501,"threads_started":1}
W20260806 01:01:39.907234 19212 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.256s user 0.004s sys 0.023s
W20260806 01:01:39.909168 19211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:38.629732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:48623 (local address 127.5.108.190:36523)
0806 01:01:38.630308 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:38.630314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:38.631754 (+ 1440us) server_negotiation.cc:407] Connection header received
0806 01:01:38.635524 (+ 3770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:38.635532 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:38.635620 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:38.635792 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:38.638742 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:38.644709 (+ 5967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:38.648289 (+ 3580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:38.649282 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:38.649443 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.908732 (+1259289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:48623: BlockingRecv error: recv got EOF from 127.5.108.130:48623 (error 108)
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":295,"threads_started":1}
I20260806 01:01:39.910316 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:38.630248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:48623)
0806 01:01:38.631530 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:38.631598 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:38.635106 (+ 3508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:38.636189 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:38.636206 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:38.637378 (+ 1172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:38.638439 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:38.638474 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:38.644992 (+ 6518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:38.645001 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:38.647998 (+ 2997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:38.648032 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:38.649913 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:38.649922 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:38.650359 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:38.651288 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:38.651331 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:39.908561 (+1257230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":465,"threads_started":1}
W20260806 01:01:39.911360 19208 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:36523 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.108.190:36523: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:39.925540 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:39.912077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:36847)
0806 01:01:39.912338 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:39.912368 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:39.912530 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:39.912982 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:39.912989 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:39.913668 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:39.913995 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.914011 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.915104 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.915113 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.917697 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.917724 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.918541 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.918551 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.918835 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.919740 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:39.919776 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:39.925206 (+ 5430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 01:01:39.925678 19211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:39.912294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:36847 (local address 127.5.108.190:36523)
0806 01:01:39.912463 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:39.912467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:39.912484 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:01:39.912699 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:39.912707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:39.912799 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:39.912910 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:39.914183 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.914944 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.917901 (+ 2957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.918312 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.918410 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.925423 (+ 7013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:36847: BlockingRecv error: recv got EOF from 127.5.108.130:36847 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:39 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:39.939172 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:39.926299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:41739)
0806 01:01:39.926460 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:39.926484 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:39.926606 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:39.927413 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:39.927419 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:39.927721 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:39.927887 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.927895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.929958 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.929962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.930936 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.930948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.932597 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.932603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.932826 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.933584 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:39.933617 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:39.938702 (+ 5085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:01:39.939220 19211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:39.926677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:41739 (local address 127.5.108.190:36523)
0806 01:01:39.926931 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:39.926939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:39.926965 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:01:39.927044 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:39.927053 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:39.927187 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:39.927352 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:39.928045 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.929778 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.931088 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.932131 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.932306 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.938926 (+ 6620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:41739: BlockingRecv error: recv got EOF from 127.5.108.130:41739 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 01:01:39.939747 19208 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 01:01:40 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:40 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:01:40.359587 19211 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:40.348134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:37369 (local address 127.5.108.190:36523)
0806 01:01:40.348396 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:40.348406 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:40.349207 (+ 801us) server_negotiation.cc:407] Connection header received
0806 01:01:40.349425 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:40.349432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:40.349546 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:40.349659 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:40.350778 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.351608 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.352990 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.353550 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.353708 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:40.359344 (+ 5636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:37369: BlockingRecv error: recv got EOF from 127.5.108.129:37369 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:01:40.359707 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:40.348078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:37369)
0806 01:01:40.349040 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:40.349070 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:40.349262 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:40.349826 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:40.349832 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:40.350401 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:40.350594 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:40.350610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.351774 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.351783 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:40.352826 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:40.352839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.353762 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.353768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:40.353977 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:40.354677 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:40.354698 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:40.359168 (+ 4470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":593,"threads_started":1}
W20260806 01:01:40.609040 19408 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.103s user 0.006s sys 0.023s
W20260806 01:01:40.610123 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:39.489090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:53779 (local address 127.5.108.190:36523)
0806 01:01:39.489376 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:39.489385 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:39.490609 (+ 1224us) server_negotiation.cc:407] Connection header received
0806 01:01:39.494460 (+ 3851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:39.494464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:39.494583 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:39.494705 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:39.497533 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.499215 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.502826 (+ 3611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.503912 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.504051 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:40.609919 (+1105868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:53779: BlockingRecv error: recv got EOF from 127.5.108.131:53779 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 01:01:40.610973 19408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:39.489333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:53779)
0806 01:01:39.490370 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:39.490448 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:39.494082 (+ 3634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:39.495079 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:39.495097 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:39.496164 (+ 1067us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:39.497233 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.497264 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.499567 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.499576 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.502557 (+ 2981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:39.502581 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:39.504574 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:39.504584 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:39.505023 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:39.506066 (+ 1043us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:39.506119 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:40.609774 (+1103655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":370,"threads_started":1}
W20260806 01:01:40.611915 19406 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:36523 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.108.190:36523: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Aug 06 01:01:40 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:40 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:40.620723 19408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:40.612554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:47481)
0806 01:01:40.612751 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:40.612772 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:40.612877 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:40.613151 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:40.613153 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:40.613472 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:40.613622 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:40.613630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.614582 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.614585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:40.615574 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:40.615585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.616310 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.616313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:40.616428 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:40.617059 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:40.617076 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:40.620447 (+ 3371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 01:01:40.620872 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:40.612640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:47481 (local address 127.5.108.190:36523)
0806 01:01:40.612855 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:40.612859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:40.612875 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:01:40.612971 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:40.612975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:40.613044 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:40.613115 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:40.613753 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.614475 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.615722 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.616158 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.616268 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:40.620632 (+ 4364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:47481: BlockingRecv error: recv got EOF from 127.5.108.131:47481 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
Aug 06 01:01:40 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:40 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:40.630306 19408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:40.621391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:34961)
0806 01:01:40.621588 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:40.621602 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:40.621674 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:40.621978 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:40.621981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:40.622394 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:40.622602 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:40.622616 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.623930 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.623935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:40.625426 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:40.625440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.626105 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.626112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:40.626295 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:40.626830 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:40.626856 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:40.630034 (+ 3178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:01:40.630441 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:40.621485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:34961 (local address 127.5.108.190:36523)
0806 01:01:40.621637 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:40.621641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:40.621656 (+ 15us) server_negotiation.cc:407] Connection header received
0806 01:01:40.621768 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:40.621771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:40.621848 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:40.621944 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:40.622742 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.623796 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.625563 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.625943 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.626047 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:40.630222 (+ 4175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:34961: BlockingRecv error: recv got EOF from 127.5.108.131:34961 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 01:01:40.630725 19406 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 01:01:40 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:40 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:40.949262 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:40.940449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:36373)
0806 01:01:40.941036 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:40.941052 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:40.941201 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:40.941581 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:40.941584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:40.942009 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:40.942157 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:40.942166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.943107 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.943110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:40.944078 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:40.944089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.944775 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.944779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:40.944898 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:40.945454 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:40.945471 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:40.949016 (+ 3545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":345,"threads_started":1}
W20260806 01:01:40.949519 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:40.940484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:36373 (local address 127.5.108.190:36523)
0806 01:01:40.940688 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:40.940694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:40.941133 (+ 439us) server_negotiation.cc:407] Connection header received
0806 01:01:40.941366 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:40.941371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:40.941450 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:40.941528 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:40.942284 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.942988 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.944206 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:40.944619 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:40.944717 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:40.949228 (+ 4511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:36373: BlockingRecv error: recv got EOF from 127.5.108.130:36373 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 01:01:41 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:41 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:41.369514 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:41.360716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:50973)
0806 01:01:41.361218 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:41.361233 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:41.361406 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:41.361765 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:41.361770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:41.362187 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:41.362367 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:41.362376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.363353 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.363358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:41.364269 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:41.364280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.365043 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.365049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:41.365234 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:41.365785 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:41.365800 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:41.369262 (+ 3462us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":282,"threads_started":1}
W20260806 01:01:41.369688 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:41.360791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:50973 (local address 127.5.108.190:36523)
0806 01:01:41.361021 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:41.361030 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:41.361328 (+ 298us) server_negotiation.cc:407] Connection header received
0806 01:01:41.361541 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:41.361546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:41.361618 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:41.361731 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:41.362543 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.363216 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.364425 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.364884 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.365022 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:41.369451 (+ 4429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:50973: BlockingRecv error: recv got EOF from 127.5.108.129:50973 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 01:01:41 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:41 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:41.640094 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:41.631277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:45849)
0806 01:01:41.631926 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:41.631940 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:41.632103 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:41.632506 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:41.632510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:41.633008 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:41.633161 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:41.633171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.634128 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.634131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:41.635007 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:41.635017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.635694 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.635698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:41.635808 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:41.636381 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:41.636402 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:41.639783 (+ 3381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":433,"threads_started":1}
W20260806 01:01:41.640239 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:41.631370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:45849 (local address 127.5.108.190:36523)
0806 01:01:41.631610 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:41.631619 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:41.632045 (+ 426us) server_negotiation.cc:407] Connection header received
0806 01:01:41.632246 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:41.632252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:41.632352 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:41.632431 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:41.633285 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.634011 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.635124 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.635553 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.635637 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:41.639971 (+ 4334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:45849: BlockingRecv error: recv got EOF from 127.5.108.131:45849 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 01:01:41 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:41 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:41.958562 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:41.950166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:50975)
0806 01:01:41.950729 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:41.950744 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:41.950854 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:41.951099 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:41.951102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:41.951517 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:41.951712 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:41.951722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.952712 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.952716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:41.953603 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:41.953614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.954258 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.954262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:41.954380 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:41.954943 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:41.954972 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:41.958315 (+ 3343us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":329,"threads_started":1}
W20260806 01:01:41.958714 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:41.950225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:50975 (local address 127.5.108.190:36523)
0806 01:01:41.950414 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:41.950418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:41.950816 (+ 398us) server_negotiation.cc:407] Connection header received
0806 01:01:41.950947 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:41.950950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:41.951009 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:41.951086 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:41.951861 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.952568 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.953722 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:41.954132 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:41.954220 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:41.958525 (+ 4305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:50975: BlockingRecv error: recv got EOF from 127.5.108.130:50975 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 01:01:42 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:42 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:42.379762 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:42.370586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:49557)
0806 01:01:42.371195 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:42.371214 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:42.371349 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:42.371710 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:42.371715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:42.372096 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:42.372275 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:42.372283 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.373320 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.373327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:42.374238 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:42.374248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.374987 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.374991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:42.375160 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:42.375769 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:42.375785 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:42.379340 (+ 3555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"spinlock_wait_cycles":20736,"thread_start_us":337,"threads_started":1}
W20260806 01:01:42.379812 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:42.371032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:49557 (local address 127.5.108.190:36523)
0806 01:01:42.371230 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:42.371249 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:42.371268 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:01:42.371501 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:42.371506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:42.371579 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:42.371682 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:42.372425 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.373173 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.374396 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.374848 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.374976 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:42.379574 (+ 4598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:49557: BlockingRecv error: recv got EOF from 127.5.108.129:49557 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 01:01:42 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:42 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:01:42.650877 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:42.641024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:54071 (local address 127.5.108.190:36523)
0806 01:01:42.641208 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:42.641213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:42.641690 (+ 477us) server_negotiation.cc:407] Connection header received
0806 01:01:42.641861 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:42.641864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:42.641943 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:42.642018 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:42.642959 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.643735 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.645585 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.646045 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.646140 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:42.650569 (+ 4429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:54071: BlockingRecv error: recv got EOF from 127.5.108.131:54071 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:01:42.650918 19424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:42.640961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:54071)
0806 01:01:42.641579 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:42.641599 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:42.641750 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:42.642074 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:42.642079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:42.642578 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:42.642809 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:42.642820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.643877 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.643882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:42.645425 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:42.645440 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.646190 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.646195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:42.646365 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:42.646922 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:42.646942 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:42.650361 (+ 3419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":375,"threads_started":1}
Aug 06 01:01:42 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:42 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:42.969056 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:42.959705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:60401)
0806 01:01:42.960242 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:42.960262 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:42.960394 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:42.960710 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:42.960714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:42.961204 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:42.961449 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:42.961462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.962475 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.962479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:42.963508 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:42.963520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.964165 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.964169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:42.964289 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:42.964862 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:42.964878 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:42.968792 (+ 3914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":274,"threads_started":1}
W20260806 01:01:42.969122 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:42.959752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:60401 (local address 127.5.108.190:36523)
0806 01:01:42.959957 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:42.959966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:42.960348 (+ 382us) server_negotiation.cc:407] Connection header received
0806 01:01:42.960499 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:42.960502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:42.960562 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:42.960638 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:42.961625 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.962350 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.963627 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:42.964029 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:42.964127 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:42.968945 (+ 4818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:60401: BlockingRecv error: recv got EOF from 127.5.108.130:60401 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 01:01:43 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:43 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:43.395054 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:43.380865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:33117)
0806 01:01:43.381447 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:43.381462 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:43.381591 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:43.381910 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:43.381913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:43.382275 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:43.382461 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:43.382469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.383329 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.383332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:43.384361 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:43.384373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.385060 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.385064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:43.385215 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:43.391492 (+ 6277us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:43.391509 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:43.394818 (+ 3309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":332,"threads_started":1}
W20260806 01:01:43.395206 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:43.380906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:33117 (local address 127.5.108.190:36523)
0806 01:01:43.381101 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:43.381106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:43.381553 (+ 447us) server_negotiation.cc:407] Connection header received
0806 01:01:43.381685 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:43.381689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:43.381752 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:43.381869 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:43.382577 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.383224 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.384502 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.384909 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.384999 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:43.395009 (+ 10010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:33117: BlockingRecv error: recv got EOF from 127.5.108.129:33117 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 01:01:43 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:43 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:43.661595 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:43.651968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:38319)
0806 01:01:43.652489 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:43.652504 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:43.652659 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:43.653062 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:43.653066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:43.653488 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:43.653654 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:43.653663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.654778 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.654782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:43.655786 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:43.655799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.656513 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.656519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:43.656707 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:43.657391 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:43.657410 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:43.661219 (+ 3809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":258,"threads_started":1}
W20260806 01:01:43.661727 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:43.652053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:38319 (local address 127.5.108.190:36523)
0806 01:01:43.652309 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:43.652315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:43.652593 (+ 278us) server_negotiation.cc:407] Connection header received
0806 01:01:43.652836 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:43.652841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:43.652912 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:43.653023 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:43.653841 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.654585 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.655931 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.656353 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.656486 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:43.661435 (+ 4949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:38319: BlockingRecv error: recv got EOF from 127.5.108.131:38319 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 06 01:01:43 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:43 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:43.979321 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:43.970313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:56927)
0806 01:01:43.970863 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:43.970881 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:43.971000 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:43.971316 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:43.971319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:43.971730 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:43.971891 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:43.971899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.972928 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.972931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:43.973896 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:43.973918 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.974535 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.974539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:43.974661 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:43.975231 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:43.975253 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:43.979067 (+ 3814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":311,"threads_started":1}
W20260806 01:01:43.979581 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:43.970391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:56927 (local address 127.5.108.190:36523)
0806 01:01:43.970564 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:43.970568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:43.970999 (+ 431us) server_negotiation.cc:407] Connection header received
0806 01:01:43.971097 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:43.971101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:43.971170 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:43.971274 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:43.972041 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.972804 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.974025 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:43.974409 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:43.974506 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:43.979274 (+ 4768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:56927: BlockingRecv error: recv got EOF from 127.5.108.130:56927 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 01:01:44 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:44 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:44.405499 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:44.396130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:35091)
0806 01:01:44.396667 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:44.396683 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:44.396845 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:44.397235 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:44.397240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:44.397719 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:44.397885 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:44.397895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.398940 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.398944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:44.400061 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:44.400073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.400869 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.400874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:44.401046 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:44.401638 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:44.401654 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:44.405276 (+ 3622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":304,"threads_started":1}
W20260806 01:01:44.405726 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:44.396193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:35091 (local address 127.5.108.190:36523)
0806 01:01:44.396439 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:44.396447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:44.396796 (+ 349us) server_negotiation.cc:407] Connection header received
0806 01:01:44.397029 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:44.397033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:44.397094 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:44.397198 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:44.398031 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.398810 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.400199 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.400723 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.400858 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:44.405467 (+ 4609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:35091: BlockingRecv error: recv got EOF from 127.5.108.129:35091 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 01:01:44 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:44 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:44.672663 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:44.662883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:54017)
0806 01:01:44.663478 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:44.663494 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:44.663607 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:44.663932 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:44.663935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:44.664315 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:44.664495 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:44.664504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.665529 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.665532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:44.666527 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:44.666538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.667698 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.667702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:44.667911 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:44.668656 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:44.668673 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:44.672416 (+ 3743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":330,"threads_started":1}
W20260806 01:01:44.672832 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:44.662977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:54017 (local address 127.5.108.190:36523)
0806 01:01:44.663199 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:44.663203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:44.663574 (+ 371us) server_negotiation.cc:407] Connection header received
0806 01:01:44.663697 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:44.663701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:44.663786 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:44.663885 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:44.664689 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.665413 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.666713 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.667513 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.667700 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:44.672604 (+ 4904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:54017: BlockingRecv error: recv got EOF from 127.5.108.131:54017 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 01:01:44 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:44 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:44.989509 19434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:44.980355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:55833)
0806 01:01:44.980895 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:44.980912 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:44.981037 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:44.981398 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:44.981402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:44.981918 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:44.982090 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:44.982099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.983051 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.983054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:44.984079 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:44.984091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.984826 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.984831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:44.984969 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:44.985544 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:44.985561 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:44.989285 (+ 3724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":294,"threads_started":1}
W20260806 01:01:44.989717 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:44.980557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:55833 (local address 127.5.108.190:36523)
0806 01:01:44.980787 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:44.980794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:44.980985 (+ 191us) server_negotiation.cc:407] Connection header received
0806 01:01:44.981133 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:44.981136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:44.981195 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:44.981273 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:44.982220 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.982936 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.984197 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:44.984631 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:44.984721 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:44.989495 (+ 4774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:55833: BlockingRecv error: recv got EOF from 127.5.108.130:55833 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 01:01:45 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:45 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:45.417778 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:45.406824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:48403)
0806 01:01:45.407477 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:45.407501 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:45.407723 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:45.408111 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:45.408116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:45.408711 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:45.408896 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:45.408906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.409980 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.409985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:45.410974 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:45.410987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.411994 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.411999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:45.412245 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:45.413041 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:45.413069 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:45.417382 (+ 4313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":312,"threads_started":1}
W20260806 01:01:45.417917 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:45.406896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:48403 (local address 127.5.108.190:36523)
0806 01:01:45.407203 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:45.407211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:45.407603 (+ 392us) server_negotiation.cc:407] Connection header received
0806 01:01:45.407877 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:45.407882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:45.407950 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:45.408069 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:45.409070 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.409805 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.411165 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.411774 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.411924 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:45.417653 (+ 5729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:48403: BlockingRecv error: recv got EOF from 127.5.108.129:48403 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 06 01:01:45 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:45 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:45.682395 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:45.673578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:58007)
0806 01:01:45.674083 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:45.674099 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:45.674224 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:45.674494 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:45.674497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:45.674927 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:45.675076 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:45.675084 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.676039 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.676042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:45.676952 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:45.676963 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.677795 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.677800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:45.677967 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:45.678696 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:45.678720 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:45.682165 (+ 3445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":276,"threads_started":1}
W20260806 01:01:45.682559 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:45.673630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:58007 (local address 127.5.108.190:36523)
0806 01:01:45.673867 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:45.673877 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:45.674177 (+ 300us) server_negotiation.cc:407] Connection header received
0806 01:01:45.674311 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:45.674315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:45.674373 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:45.674449 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:45.675197 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.675930 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.677064 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.677631 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.677761 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:45.682343 (+ 4582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:58007: BlockingRecv error: recv got EOF from 127.5.108.131:58007 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 01:01:45 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:45 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:46.000150 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:45.990479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:49551)
0806 01:01:45.991035 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:45.991051 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:45.991168 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:45.991506 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:45.991509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:45.991953 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:45.992194 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:45.992213 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.993413 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.993417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:45.994383 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:45.994394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.995041 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.995045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:45.995173 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:45.995746 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:45.995763 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:45.999814 (+ 4051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":322,"threads_started":1}
W20260806 01:01:46.000257 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:45.990556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:49551 (local address 127.5.108.190:36523)
0806 01:01:45.990753 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:45.990757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:45.991126 (+ 369us) server_negotiation.cc:407] Connection header received
0806 01:01:45.991275 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:45.991278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:45.991337 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:45.991434 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:45.992423 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.993253 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.994509 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:45.994895 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:45.994984 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:46.000015 (+ 5031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:49551: BlockingRecv error: recv got EOF from 127.5.108.130:49551 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 01:01:46 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:46 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:46.430539 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:46.418899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:33103)
0806 01:01:46.419736 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:46.419756 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:46.419901 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:46.420253 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:46.420260 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:46.420896 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:46.421179 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:46.421206 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:46.422445 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:46.422453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:46.424389 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:46.424407 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:46.425140 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:46.425148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:46.425384 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:46.426044 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:46.426075 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:46.430168 (+ 4093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":449,"threads_started":1}
W20260806 01:01:46.430601 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:46.418953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:33103 (local address 127.5.108.190:36523)
0806 01:01:46.419158 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:46.419165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:46.419855 (+ 690us) server_negotiation.cc:407] Connection header received
0806 01:01:46.420026 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:46.420031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:46.420088 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:46.420191 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:46.421348 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:46.422301 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:46.424547 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:46.424974 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:46.425065 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:46.430380 (+ 5315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:33103: BlockingRecv error: recv got EOF from 127.5.108.129:33103 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 01:01:46 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:46 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:46.692284 19442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:46.683359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:60901)
0806 01:01:46.683936 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:46.683954 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:46.684080 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:46.684392 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:46.684395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:46.684793 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:46.684976 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:46.684984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:46.685925 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:46.685928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:46.687002 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:46.687015 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:46.687744 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:46.687749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:46.687870 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:46.688401 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:46.688418 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:46.692057 (+ 3639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":308,"threads_started":1}
W20260806 01:01:46.692456 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:46.683538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:60901 (local address 127.5.108.190:36523)
0806 01:01:46.683712 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:46.683717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:46.684037 (+ 320us) server_negotiation.cc:407] Connection header received
0806 01:01:46.684184 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:46.684188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:46.684248 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:46.684343 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:46.685095 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:46.685805 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:46.687142 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:46.687615 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:46.687705 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:46.692234 (+ 4529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:60901: BlockingRecv error: recv got EOF from 127.5.108.131:60901 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 01:01:47 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:47 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:47.010486 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:47.001167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:35633)
0806 01:01:47.001776 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:47.001791 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:47.001943 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:47.002263 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:47.002267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:47.002707 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:47.002892 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:47.002902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.003976 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.003987 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:47.004997 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:47.005010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.005789 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.005795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:47.005964 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:47.006538 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:47.006555 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:47.010161 (+ 3606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":291,"threads_started":1}
W20260806 01:01:47.010627 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:47.001228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:35633 (local address 127.5.108.190:36523)
0806 01:01:47.001493 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:47.001500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:47.001875 (+ 375us) server_negotiation.cc:407] Connection header received
0806 01:01:47.002059 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:47.002069 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:47.002128 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:47.002228 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:47.003037 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.003846 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.005151 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.005624 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.005774 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:47.010358 (+ 4584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:35633: BlockingRecv error: recv got EOF from 127.5.108.130:35633 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 06 01:01:47 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:47 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:47.440827 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:47.431653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:48971)
0806 01:01:47.432169 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:47.432185 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:47.432298 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:47.432571 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:47.432574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:47.433004 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:47.433182 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:47.433190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.434213 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.434216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:47.435140 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:47.435150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.436032 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.436037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:47.436188 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:47.436887 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:47.436903 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:47.440575 (+ 3672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":263,"threads_started":1}
W20260806 01:01:47.441045 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:47.431759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:48971 (local address 127.5.108.190:36523)
0806 01:01:47.432002 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:47.432020 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:47.432265 (+ 245us) server_negotiation.cc:407] Connection header received
0806 01:01:47.432381 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:47.432385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:47.432446 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:47.432523 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:47.433351 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.434092 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.435285 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.435729 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.435889 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:47.440808 (+ 4919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:48971: BlockingRecv error: recv got EOF from 127.5.108.129:48971 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 06 01:01:47 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:47 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:47.702538 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:47.693355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:37235)
0806 01:01:47.693925 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:47.693941 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:47.694058 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:47.694355 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:47.694358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:47.694776 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:47.694949 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:47.694957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.696134 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.696138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:47.697559 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:47.697575 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.698315 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.698319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:47.698432 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:47.699005 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:47.699021 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:47.702290 (+ 3269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":275,"threads_started":1}
W20260806 01:01:47.702765 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:47.693528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:37235 (local address 127.5.108.190:36523)
0806 01:01:47.693806 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:47.693812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:47.694026 (+ 214us) server_negotiation.cc:407] Connection header received
0806 01:01:47.694159 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:47.694163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:47.694224 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:47.694304 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:47.695074 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.695961 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.697699 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:47.698183 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:47.698276 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:47.702505 (+ 4229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:37235: BlockingRecv error: recv got EOF from 127.5.108.131:37235 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 06 01:01:48 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:48 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:48.020943 19449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:48.011473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:33349)
0806 01:01:48.012045 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:48.012062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:48.012179 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:48.012486 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:48.012489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:48.012908 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:48.013088 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:48.013096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.014223 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.014226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:48.015633 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:48.015646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.016529 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.016534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:48.016673 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:48.017293 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:48.017310 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:48.020694 (+ 3384us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":297,"threads_started":1}
W20260806 01:01:48.021170 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:48.011549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:33349 (local address 127.5.108.190:36523)
0806 01:01:48.011755 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:48.011760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:48.012136 (+ 376us) server_negotiation.cc:407] Connection header received
0806 01:01:48.012284 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:48.012288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:48.012349 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:48.012434 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:48.013227 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.014090 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.015766 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.016349 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.016458 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:48.020894 (+ 4436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:33349: BlockingRecv error: recv got EOF from 127.5.108.130:33349 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 01:01:48 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:48 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:48.454802 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:48.441787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:44375)
0806 01:01:48.442354 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:48.442370 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:48.442525 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:48.442869 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:48.442874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:48.443333 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:48.443513 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:48.443522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.444557 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.444563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:48.445506 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:48.445517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.446211 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.446217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:48.446396 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:48.451036 (+ 4640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:48.451072 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:48.454482 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":335,"threads_started":1}
W20260806 01:01:48.454936 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:48.441845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:44375 (local address 127.5.108.190:36523)
0806 01:01:48.442075 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:48.442083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:48.442461 (+ 378us) server_negotiation.cc:407] Connection header received
0806 01:01:48.442648 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:48.442653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:48.442722 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:48.442832 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:48.443663 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.444420 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.445664 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.446053 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.446178 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:48.454675 (+ 8497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:44375: BlockingRecv error: recv got EOF from 127.5.108.129:44375 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 01:01:48 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:48 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:48.713076 19454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:48.703650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:55859)
0806 01:01:48.704203 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:48.704220 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:48.704354 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:48.704662 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:48.704666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:48.705155 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:48.705334 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:48.705343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.706326 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.706330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:48.707355 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:48.707403 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.708348 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.708354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:48.708482 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:48.709058 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:48.709075 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:48.712825 (+ 3750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":291,"threads_started":1}
W20260806 01:01:48.713255 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:48.703748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:55859 (local address 127.5.108.190:36523)
0806 01:01:48.703952 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:48.703965 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:48.704296 (+ 331us) server_negotiation.cc:407] Connection header received
0806 01:01:48.704440 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:48.704443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:48.704516 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:48.704596 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:48.705461 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.706200 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.707545 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:48.708105 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:48.708233 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:48.713021 (+ 4788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:55859: BlockingRecv error: recv got EOF from 127.5.108.131:55859 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 01:01:49 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:49 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:49.031323 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:49.021940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:44943)
0806 01:01:49.022502 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:49.022518 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:49.022668 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:49.023023 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:49.023028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:49.023591 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:49.023759 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:49.023768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.024805 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.024811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:49.025768 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:49.025779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.026553 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.026559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:49.026749 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:49.027300 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:49.027317 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:49.031021 (+ 3704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":298,"threads_started":1}
W20260806 01:01:49.031453 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:49.021974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:44943 (local address 127.5.108.190:36523)
0806 01:01:49.022261 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:49.022270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:49.022620 (+ 350us) server_negotiation.cc:407] Connection header received
0806 01:01:49.022803 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:49.022808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:49.022879 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:49.022982 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:49.023906 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.024659 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.025915 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.026382 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.026536 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:49.031204 (+ 4668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:44943: BlockingRecv error: recv got EOF from 127.5.108.130:44943 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 01:01:49 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:49 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:49.465216 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:49.455905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:57657)
0806 01:01:49.456624 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:49.456640 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:49.456760 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:49.457144 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:49.457149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:49.457555 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:49.457717 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:49.457726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.458781 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.458787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:49.459817 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:49.459828 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.460578 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.460585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:49.460768 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:49.461436 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:49.461454 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:49.464962 (+ 3508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":479,"threads_started":1}
W20260806 01:01:49.465427 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:49.455982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:57657 (local address 127.5.108.190:36523)
0806 01:01:49.456201 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:49.456210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:49.456725 (+ 515us) server_negotiation.cc:407] Connection header received
0806 01:01:49.456916 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:49.456921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:49.456998 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:49.457106 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:49.457882 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.458628 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.459963 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.460414 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.460546 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:49.465138 (+ 4592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:57657: BlockingRecv error: recv got EOF from 127.5.108.129:57657 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 01:01:49 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:49 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:49.722749 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:49.714222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:36177)
0806 01:01:49.714811 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:49.714826 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:49.714974 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:49.715283 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:49.715286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:49.715682 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:49.715825 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:49.715833 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.716731 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.716734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:49.717650 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:49.717661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.718327 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.718332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:49.718459 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:49.719013 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:49.719029 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:49.722497 (+ 3468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":314,"threads_started":1}
W20260806 01:01:49.722958 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:49.714257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:36177 (local address 127.5.108.190:36523)
0806 01:01:49.714493 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:49.714502 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:49.714931 (+ 429us) server_negotiation.cc:407] Connection header received
0806 01:01:49.715083 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:49.715088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:49.715157 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:49.715230 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:49.715948 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.716622 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.717779 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:49.718195 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:49.718291 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:49.722702 (+ 4411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:36177: BlockingRecv error: recv got EOF from 127.5.108.131:36177 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 01:01:50 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:50 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:50.042033 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:50.032471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:50571)
0806 01:01:50.033113 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:50.033131 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:50.033305 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:50.033746 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:50.033751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:50.034202 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:50.034455 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:50.034467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.035509 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.035513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:50.036677 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:50.036690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.037553 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.037562 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:50.037702 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:50.038287 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:50.038306 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:50.041787 (+ 3481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":342,"threads_started":1}
W20260806 01:01:50.042181 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:50.032547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:50571 (local address 127.5.108.190:36523)
0806 01:01:50.032879 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:50.032885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:50.033231 (+ 346us) server_negotiation.cc:407] Connection header received
0806 01:01:50.033460 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:50.033464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:50.033568 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:50.033674 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:50.034599 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.035290 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.036814 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.037341 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.037443 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:50.041958 (+ 4515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:50571: BlockingRecv error: recv got EOF from 127.5.108.130:50571 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 06 01:01:50 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:50 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:50.474534 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:50.466137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:56239)
0806 01:01:50.466705 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:50.466722 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:50.466834 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:50.467122 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:50.467125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:50.467636 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:50.467830 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:50.467839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.468861 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.468865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:50.469759 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:50.469769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.470387 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.470391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:50.470511 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:50.471082 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:50.471098 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:50.474321 (+ 3223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":322,"threads_started":1}
W20260806 01:01:50.474710 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:50.466306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:56239 (local address 127.5.108.190:36523)
0806 01:01:50.466511 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:50.466519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:50.466806 (+ 287us) server_negotiation.cc:407] Connection header received
0806 01:01:50.466934 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:50.466938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:50.466999 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:50.467079 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:50.467963 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.468750 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.469881 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.470260 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.470348 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:50.474485 (+ 4137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:56239: BlockingRecv error: recv got EOF from 127.5.108.129:56239 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 01:01:50 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:50 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:50.732658 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:50.723894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:39889)
0806 01:01:50.724490 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:50.724506 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:50.724621 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:50.724991 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:50.724995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:50.725377 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:50.725542 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:50.725551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.726565 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.726568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:50.727597 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:50.727609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.728384 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.728388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:50.728499 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:50.729083 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:50.729113 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:50.732418 (+ 3305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":332,"threads_started":1}
W20260806 01:01:50.732879 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:50.723991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:39889 (local address 127.5.108.190:36523)
0806 01:01:50.724183 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:50.724188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:50.724581 (+ 393us) server_negotiation.cc:407] Connection header received
0806 01:01:50.724774 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:50.724779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:50.724849 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:50.724948 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:50.725685 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.726438 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.727767 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:50.728240 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:50.728355 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:50.732614 (+ 4259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:39889: BlockingRecv error: recv got EOF from 127.5.108.131:39889 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 01:01:51 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:51 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:51.051694 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:51.043053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:43903)
0806 01:01:51.043579 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:51.043595 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:51.043709 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:51.044013 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:51.044016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:51.044453 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:51.044642 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:51.044651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.045632 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.045635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:51.046510 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:51.046520 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.047156 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.047160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:51.047292 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:51.048113 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:51.048129 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:51.051484 (+ 3355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":290,"threads_started":1}
W20260806 01:01:51.051877 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:51.043114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:43903 (local address 127.5.108.190:36523)
0806 01:01:51.043390 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:51.043397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:51.043699 (+ 302us) server_negotiation.cc:407] Connection header received
0806 01:01:51.043808 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:51.043811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:51.043873 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:51.043956 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:51.044769 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.045509 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.046626 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.047035 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.047119 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:51.051637 (+ 4518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:43903: BlockingRecv error: recv got EOF from 127.5.108.130:43903 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 01:01:51 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:51 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:51.487088 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:51.475531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:49415)
0806 01:01:51.476080 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:51.476118 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:51.476268 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:51.476751 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:51.476756 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:51.477263 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:51.477439 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:51.477447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.478507 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.478511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:51.479511 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:51.479523 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.480210 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.480214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:51.480345 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:51.480941 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:51.480956 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:51.486755 (+ 5799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":314,"threads_started":1}
W20260806 01:01:51.487211 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:51.475568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:49415 (local address 127.5.108.190:36523)
0806 01:01:51.475763 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:51.475772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:51.476211 (+ 439us) server_negotiation.cc:407] Connection header received
0806 01:01:51.476417 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:51.476421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:51.476501 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:51.476660 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:51.477557 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.478387 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.479628 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.480063 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.480152 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:51.486929 (+ 6777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:49415: BlockingRecv error: recv got EOF from 127.5.108.129:49415 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 01:01:51 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:51 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:51.742844 19467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:51.733753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:48755)
0806 01:01:51.734435 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:51.734455 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:51.734598 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:51.734968 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:51.734973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:51.735579 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:51.735773 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:51.735783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.736862 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.736866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:51.737866 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:51.737880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.738590 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.738594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:51.738715 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:51.739422 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:51.739448 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:51.742635 (+ 3187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":422,"threads_started":1}
W20260806 01:01:51.743073 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:51.733832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:48755 (local address 127.5.108.190:36523)
0806 01:01:51.734104 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:51.734113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:51.734545 (+ 432us) server_negotiation.cc:407] Connection header received
0806 01:01:51.734746 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:51.734752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:51.734822 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:51.734919 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:51.735905 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.736748 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.738009 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:51.738462 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:51.738557 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:51.742814 (+ 4257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:48755: BlockingRecv error: recv got EOF from 127.5.108.131:48755 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 01:01:52 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:52 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:52.062125 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:52.052868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:50245)
0806 01:01:52.053546 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:52.053563 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:52.053711 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:52.054123 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:52.054128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:52.054573 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:52.054759 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:52.054768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.055913 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.055936 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:52.056855 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:52.056866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.057623 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.057629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:52.057814 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:52.058463 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:52.058480 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:52.061774 (+ 3294us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":340,"threads_started":1}
W20260806 01:01:52.062250 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:52.052952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:50245 (local address 127.5.108.190:36523)
0806 01:01:52.053176 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:52.053184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:52.053650 (+ 466us) server_negotiation.cc:407] Connection header received
0806 01:01:52.053884 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:52.053888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:52.053987 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:52.054085 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:52.054926 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.055768 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.057001 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.057445 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.057581 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:52.061973 (+ 4392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:50245: BlockingRecv error: recv got EOF from 127.5.108.130:50245 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 01:01:52 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:52 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:52.497839 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:52.488080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:44831)
0806 01:01:52.488588 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:52.488604 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:52.488774 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:52.489147 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:52.489152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:52.489594 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:52.489742 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:52.489751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.490815 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.490820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:52.492030 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:52.492045 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.493024 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.493028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:52.493158 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:52.493799 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:52.493816 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:52.497487 (+ 3671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":262,"threads_started":1}
W20260806 01:01:52.497951 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:52.488176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:44831 (local address 127.5.108.190:36523)
0806 01:01:52.488408 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:52.488415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:52.488727 (+ 312us) server_negotiation.cc:407] Connection header received
0806 01:01:52.488930 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:52.488934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:52.489004 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:52.489112 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:52.489883 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.490672 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.492200 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.492894 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.492992 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:52.497691 (+ 4699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:44831: BlockingRecv error: recv got EOF from 127.5.108.129:44831 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 01:01:52 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:52 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:52.753057 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:52.743743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:60247)
0806 01:01:52.744270 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:52.744286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:52.744417 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:52.744711 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:52.744715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:52.745129 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:52.745303 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:52.745312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.746301 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.746304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:52.747256 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:52.747267 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.748044 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.748048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:52.748194 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:52.748830 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:52.748847 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:52.752810 (+ 3963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":283,"threads_started":1}
W20260806 01:01:52.753320 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:52.743819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:60247 (local address 127.5.108.190:36523)
0806 01:01:52.744053 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:52.744061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:52.744376 (+ 315us) server_negotiation.cc:407] Connection header received
0806 01:01:52.744514 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:52.744518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:52.744578 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:52.744665 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:52.745428 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.746184 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.747457 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:52.747888 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:52.747985 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:52.753032 (+ 5047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:60247: BlockingRecv error: recv got EOF from 127.5.108.131:60247 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 01:01:53 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:53 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:53.072187 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:53.063089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:34107)
0806 01:01:53.063614 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:53.063629 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:53.063741 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:53.064056 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:53.064059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:53.064517 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:53.064680 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:53.064688 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.065684 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.065687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:53.066589 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:53.066601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.067313 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.067317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:53.067512 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:53.068069 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:53.068085 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:53.071959 (+ 3874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":301,"threads_started":1}
W20260806 01:01:53.072364 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:53.063289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:34107 (local address 127.5.108.190:36523)
0806 01:01:53.063497 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:53.063502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:53.063704 (+ 202us) server_negotiation.cc:407] Connection header received
0806 01:01:53.063865 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:53.063869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:53.063928 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:53.064011 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:53.064809 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.065545 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.066731 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.067191 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.067297 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:53.072136 (+ 4839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:34107: BlockingRecv error: recv got EOF from 127.5.108.130:34107 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 01:01:53 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:53 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.129@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:01:53.514387 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:53.499264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:55139 (local address 127.5.108.190:36523)
0806 01:01:53.499560 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:53.499567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:53.499840 (+ 273us) server_negotiation.cc:407] Connection header received
0806 01:01:53.499964 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:53.499968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:53.500030 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:53.500113 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:53.500863 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.501585 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.502782 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.503238 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.503343 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:53.514203 (+ 10860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:55139: BlockingRecv error: recv got EOF from 127.5.108.129:55139 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:01:53.514478 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:53.499179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.129:55139)
0806 01:01:53.499719 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:53.499736 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:53.499861 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:53.500157 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:53.500160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:53.500541 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:53.500728 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:53.500736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.501712 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.501715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:53.502633 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:53.502643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.503409 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.503416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:53.503624 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:53.509158 (+ 5534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:53.509187 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:53.514022 (+ 4835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":327,"threads_started":1}
Aug 06 01:01:53 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:53 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.131@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:53.763082 19476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:53.754077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.131:50419)
0806 01:01:53.754678 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:53.754694 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:53.754812 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:53.755095 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:53.755098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:53.755585 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:53.755760 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:53.755769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.756832 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.756835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:53.757774 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:53.757785 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.758669 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.758677 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:53.758864 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:53.759621 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:53.759645 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:53.762848 (+ 3203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":352,"threads_started":1}
W20260806 01:01:53.763260 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:53.754151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:50419 (local address 127.5.108.190:36523)
0806 01:01:53.754388 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:53.754393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:53.754771 (+ 378us) server_negotiation.cc:407] Connection header received
0806 01:01:53.754905 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:53.754909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:53.754967 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:53.755048 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:53.755883 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.756690 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.757952 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:53.758494 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:53.758614 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:53.763036 (+ 4422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.131:50419: BlockingRecv error: recv got EOF from 127.5.108.131:50419 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 06 01:01:54 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:01:54 dist-test-slave-x4qh krb5kdc[17903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.108.130@KRBTEST.COM for oryx/127.5.108.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:01:54.082773 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:54.073611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36523 (local address 127.5.108.130:35183)
0806 01:01:54.074164 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:54.074180 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:54.074293 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:54.074591 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:54.074594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:54.074959 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:54.075149 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:54.075157 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:54.076155 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:54.076158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:54.077113 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:54.077124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:54.077774 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:54.077777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:54.077898 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:54.078606 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:01:54.078629 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:54.082524 (+ 3895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.108.190:36523: Server oryx/127.5.108.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":304,"threads_started":1}
W20260806 01:01:54.082935 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:54.073776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:35183 (local address 127.5.108.190:36523)
0806 01:01:54.073941 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:54.073945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:54.074262 (+ 317us) server_negotiation.cc:407] Connection header received
0806 01:01:54.074405 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:54.074408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:54.074467 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:54.074545 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:54.075262 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:54.076036 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:54.077238 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:54.077650 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:54.077741 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:54.082706 (+ 4965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.130:35183: BlockingRecv error: recv got EOF from 127.5.108.130:35183 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 01:01:54.505363 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 18807
W20260806 01:01:54.519418 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 18807: Not authorized: Client connection negotiation failed: client connection to 127.5.108.129:40397: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.108.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:01:54.519642 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 18807
W20260806 01:01:54.548664 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:54.515590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:33863 (local address 127.5.108.190:36523)
0806 01:01:54.515851 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:54.515859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:54.516296 (+ 437us) server_negotiation.cc:407] Connection header received
0806 01:01:54.516508 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:54.516517 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:54.516606 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:54.516689 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:54.517983 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:54.518755 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:54.548458 (+ 29703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.108.129:33863: BlockingRecv error: recv got EOF from 127.5.108.129:33863 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 01:01:54.548621 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 19026
I20260806 01:01:54.561246 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:54.549811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.5.108.130:45749)
0806 01:01:54.550514 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:54.550523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:54.550559 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:01:54.550633 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:54.550653 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:54.550987 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:54.551224 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:54.552581 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:54.553892 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:54.555060 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:54.555714 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:54.555811 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:54.558132 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:54.558141 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:54.558155 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:54.558195 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:54.560727 (+ 2532us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.108.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 01:01:54.561000 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33364: Request ticket server kudu/127.5.108.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":338,"threads_started":1}
W20260806 01:01:54.561364 19483 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33364: Request ticket server kudu/127.5.108.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 01:01:54.561348 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 19026: Not authorized: Client connection negotiation failed: client connection to 127.5.108.130:45749: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.108.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:01:54.561522 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 19026
I20260806 01:01:54.584213 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 19217
I20260806 01:01:54.597337 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:54.585418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46470 (local address 127.5.108.131:34147)
0806 01:01:54.586089 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:54.586096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:54.586122 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:01:54.586186 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:54.586192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:54.586423 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:54.586638 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:54.588138 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:54.589173 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:54.590394 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:54.590950 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:54.591097 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:54.593808 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:54.593816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:54.593830 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:01:54.593873 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:54.596594 (+ 2721us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.108.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 01:01:54.596953 (+ 359us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46470: Request ticket server kudu/127.5.108.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":349,"threads_started":1}
W20260806 01:01:54.597362 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 19217: Not authorized: Client connection negotiation failed: client connection to 127.5.108.131:34147: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.108.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 01:01:54.597450 19484 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46470: Request ticket server kudu/127.5.108.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:01:54.597527 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 19217
I20260806 01:01:54.623924 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 18700
I20260806 01:01:54.625363 18795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:54.742369 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 18700
2026-08-06T01:01:54Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21757 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T01:01:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:54Z Disabled control of system clock
I20260806 01:01:54.838775 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36271
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 01:01:55.173806 19492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:55.174067 19492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:55.174144 19492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:55.181023 19492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:55.181115 19492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:55.181161 19492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:55.181221 19492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:55.181264 19492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:55.189945 19492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:55.191092 19492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:55.192844 19492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:55.204806 19503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:55.204831 19501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:55.204948 19500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:55.205554 19492 server_base.cc:1034] running on GCE node
I20260806 01:01:55.206368 19492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:55.207753 19492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:55.208981 19492 hybrid_clock.cc:648] HybridClock initialized: now 1785978115208943 us; error 49 us; skew 500 ppm
I20260806 01:01:55.209328 19492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:55.211599 19492 webserver.cc:460] Webserver started at http://127.5.108.190:45063/ using document root <none> and password file <none>
I20260806 01:01:55.212291 19492 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:55.212693 19492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:55.213107 19492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:55.216055 19492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "5e649500fbbe46b9a6208a2555b465c5"
format_stamp: "Formatted at 2026-08-06 01:01:55 on dist-test-slave-x4qh"
server_key: "1973a330192e6c0437705abebe0a696e"
server_key_iv: "5d290ad6c6efa4a98f5c875d2530f400"
server_key_version: "encryptionkey@0"
I20260806 01:01:55.216845 19492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "5e649500fbbe46b9a6208a2555b465c5"
format_stamp: "Formatted at 2026-08-06 01:01:55 on dist-test-slave-x4qh"
server_key: "1973a330192e6c0437705abebe0a696e"
server_key_iv: "5d290ad6c6efa4a98f5c875d2530f400"
server_key_version: "encryptionkey@0"
I20260806 01:01:55.221757 19492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:01:55.224961 19510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:55.226222 19492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 01:01:55.226403 19492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "5e649500fbbe46b9a6208a2555b465c5"
format_stamp: "Formatted at 2026-08-06 01:01:55 on dist-test-slave-x4qh"
server_key: "1973a330192e6c0437705abebe0a696e"
server_key_iv: "5d290ad6c6efa4a98f5c875d2530f400"
server_key_version: "encryptionkey@0"
I20260806 01:01:55.226614 19492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:55.247485 19492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:55.248245 19492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:55.248553 19492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:55.266844 19492 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36271
I20260806 01:01:55.266883 19584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36271 every 8 connection(s)
I20260806 01:01:55.268080 19492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:55.271776 19585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:55.273361 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 19492
I20260806 01:01:55.273554 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:55.274221 5554 external_mini_cluster.cc:1428] Setting key 3359891a3304462e1d5a709494204344
I20260806 01:01:55.284912 19585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5: Bootstrap starting.
I20260806 01:01:55.288897 19585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:55.289824 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:55.276236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.5.108.190:36271)
0806 01:01:55.277285 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:55.277298 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:55.277350 (+ 52us) server_negotiation.cc:407] Connection header received
0806 01:01:55.279765 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:55.279802 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:55.280329 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:55.280751 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:55.281270 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:55.283247 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:55.284594 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:55.285581 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:55.285768 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:55.286947 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:55.286961 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:55.286981 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:55.287699 (+ 718us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:55.287828 (+ 129us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:55.287857 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:55.287982 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:55.288202 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 01:01:55.288501 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":306,"threads_started":1}
I20260806 01:01:55.290254 19585 log.cc:824] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:55.293076 19585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5: No bootstrap required, opened a new log
I20260806 01:01:55.300348 19585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e649500fbbe46b9a6208a2555b465c5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36271 } }
I20260806 01:01:55.300665 19585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:55.300786 19585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e649500fbbe46b9a6208a2555b465c5, State: Initialized, Role: FOLLOWER
I20260806 01:01:55.301321 19585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e649500fbbe46b9a6208a2555b465c5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36271 } }
I20260806 01:01:55.301548 19585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:55.301705 19585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:55.301879 19585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:55.304245 19585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e649500fbbe46b9a6208a2555b465c5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36271 } }
I20260806 01:01:55.304737 19585 leader_election.cc:302] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e649500fbbe46b9a6208a2555b465c5; no voters:
I20260806 01:01:55.305289 19585 leader_election.cc:288] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:55.305442 19596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:55.306484 19596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [term 1 LEADER]: Becoming Leader. State: Replica: 5e649500fbbe46b9a6208a2555b465c5, State: Running, Role: LEADER
I20260806 01:01:55.306963 19596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e649500fbbe46b9a6208a2555b465c5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36271 } }
I20260806 01:01:55.307175 19585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:55.309942 19598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e649500fbbe46b9a6208a2555b465c5. Latest consensus state: current_term: 1 leader_uuid: "5e649500fbbe46b9a6208a2555b465c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e649500fbbe46b9a6208a2555b465c5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36271 } } }
I20260806 01:01:55.310441 19598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:55.310402 19597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e649500fbbe46b9a6208a2555b465c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e649500fbbe46b9a6208a2555b465c5" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36271 } } }
I20260806 01:01:55.310763 19597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:55.313582 19607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:55.317881 19607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:55.325731 19607 catalog_manager.cc:1347] Generated new cluster ID: d28cd63a555545c889a4d63cc2108fc6
I20260806 01:01:55.325848 19607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:55.344295 19607 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:55.346060 19607 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:55.361476 19607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e649500fbbe46b9a6208a2555b465c5: Generated new TSK 0
I20260806 01:01:55.362201 19607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:01:55.375471 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36271
--builtin_ntp_servers=127.5.108.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 01:01:55.734232 19624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:55.734460 19624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:55.734608 19624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:55.741524 19624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:55.741603 19624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:55.741763 19624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:55.750636 19624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:55.751983 19624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:55.753675 19624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:55.766659 19632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:55.766662 19633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:55.766899 19635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:55.767920 19624 server_base.cc:1034] running on GCE node
I20260806 01:01:55.768472 19624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:55.769313 19624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:55.770522 19624 hybrid_clock.cc:648] HybridClock initialized: now 1785978115770482 us; error 33 us; skew 500 ppm
I20260806 01:01:55.770886 19624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:55.773149 19624 webserver.cc:460] Webserver started at http://127.5.108.129:44385/ using document root <none> and password file <none>
I20260806 01:01:55.773851 19624 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:55.773962 19624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:55.774326 19624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:55.777141 19624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "0db35b7ce0d04504bfb6d967b76d9efc"
format_stamp: "Formatted at 2026-08-06 01:01:55 on dist-test-slave-x4qh"
server_key: "104e75513d688f956657348edcc1b676"
server_key_iv: "7ba1091252872de218710327f40f0e3a"
server_key_version: "encryptionkey@0"
I20260806 01:01:55.777957 19624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "0db35b7ce0d04504bfb6d967b76d9efc"
format_stamp: "Formatted at 2026-08-06 01:01:55 on dist-test-slave-x4qh"
server_key: "104e75513d688f956657348edcc1b676"
server_key_iv: "7ba1091252872de218710327f40f0e3a"
server_key_version: "encryptionkey@0"
I20260806 01:01:55.782830 19624 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260806 01:01:55.786168 19646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:55.787456 19624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:01:55.787621 19624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "0db35b7ce0d04504bfb6d967b76d9efc"
format_stamp: "Formatted at 2026-08-06 01:01:55 on dist-test-slave-x4qh"
server_key: "104e75513d688f956657348edcc1b676"
server_key_iv: "7ba1091252872de218710327f40f0e3a"
server_key_version: "encryptionkey@0"
I20260806 01:01:55.787829 19624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:55.814102 19624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:55.814905 19624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:55.815201 19624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:55.816150 19624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:55.817519 19624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:55.817651 19624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:55.817790 19624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:55.817880 19624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:55.851286 19624 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:34593
I20260806 01:01:55.851389 19819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:34593 every 8 connection(s)
I20260806 01:01:55.852543 19624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:55.858860 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 19624
I20260806 01:01:55.859051 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:01:55.859789 5554 external_mini_cluster.cc:1428] Setting key 3a645f7b1742a5bf4c7d1ea4f6eb9c5c
I20260806 01:01:55.862430 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36271
--builtin_ntp_servers=127.5.108.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 01:01:55.865458 19822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:55.855085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:49105 (local address 127.5.108.190:36271)
0806 01:01:55.855587 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:55.855591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:55.856649 (+ 1058us) server_negotiation.cc:407] Connection header received
0806 01:01:55.858408 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:55.858414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:55.858502 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:55.858627 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:55.859838 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:55.860771 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:55.862511 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:55.862955 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:55.863092 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:55.864561 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:55.864566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:55.864570 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:55.864585 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:55.864623 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:55.864635 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:55.864767 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:55.864995 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 01:01:55.865101 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":248,"threads_started":1}
I20260806 01:01:55.866542 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:55.855671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36271 (local address 127.5.108.129:49105)
0806 01:01:55.856493 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:55.856533 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:55.858140 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:55.858858 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:55.858864 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:55.858989 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:55.859665 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:55.859691 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:55.860912 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:55.860919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:55.862369 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:55.862381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:55.863195 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:55.863203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:55.863499 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:55.864217 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:55.864220 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:55.864256 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:55.864328 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:55.864748 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:55.864780 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:55.864790 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:01:55.864971 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 01:01:55.865320 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":253,"threads_started":1}
I20260806 01:01:55.868106 19820 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36271
I20260806 01:01:55.868423 19820 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:55.869287 19820 heartbeater.cc:511] Master 127.5.108.190:36271 requested a full tablet report, sending...
I20260806 01:01:55.871526 19540 ts_manager.cc:194] Registered new tserver with Master: 0db35b7ce0d04504bfb6d967b76d9efc (127.5.108.129:34593)
I20260806 01:01:55.873582 19540 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.129:49105
W20260806 01:01:56.215279 19824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:56.215649 19824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:56.215754 19824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:56.223721 19824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:56.223845 19824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:56.224022 19824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:56.235093 19824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:56.236331 19824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:56.238059 19824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:56.251259 19835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:56.251358 19833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:56.251398 19839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:56.256886 19824 server_base.cc:1034] running on GCE node
I20260806 01:01:56.257516 19824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:56.258448 19824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:56.259702 19824 hybrid_clock.cc:648] HybridClock initialized: now 1785978116259658 us; error 36 us; skew 500 ppm
I20260806 01:01:56.260017 19824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:56.262259 19824 webserver.cc:460] Webserver started at http://127.5.108.130:39715/ using document root <none> and password file <none>
I20260806 01:01:56.262959 19824 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:56.263096 19824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:56.263484 19824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:56.266225 19824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "997d537a0b8a444fac37b371ee9513ec"
format_stamp: "Formatted at 2026-08-06 01:01:56 on dist-test-slave-x4qh"
server_key: "a5a0dbfa90f10153a78b81a72c7c0bab"
server_key_iv: "c0395f4e4c9d717b4820d02f00b266b9"
server_key_version: "encryptionkey@0"
I20260806 01:01:56.267196 19824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "997d537a0b8a444fac37b371ee9513ec"
format_stamp: "Formatted at 2026-08-06 01:01:56 on dist-test-slave-x4qh"
server_key: "a5a0dbfa90f10153a78b81a72c7c0bab"
server_key_iv: "c0395f4e4c9d717b4820d02f00b266b9"
server_key_version: "encryptionkey@0"
I20260806 01:01:56.272502 19824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 01:01:56.276569 19850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:56.278008 19824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:01:56.278213 19824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "997d537a0b8a444fac37b371ee9513ec"
format_stamp: "Formatted at 2026-08-06 01:01:56 on dist-test-slave-x4qh"
server_key: "a5a0dbfa90f10153a78b81a72c7c0bab"
server_key_iv: "c0395f4e4c9d717b4820d02f00b266b9"
server_key_version: "encryptionkey@0"
I20260806 01:01:56.278496 19824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:56.306176 19824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:56.306960 19824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:56.307307 19824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:56.308331 19824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:56.309782 19824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:56.309870 19824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:56.309947 19824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:56.310001 19824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:56.343500 19824 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:42533
I20260806 01:01:56.343542 20020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:42533 every 8 connection(s)
I20260806 01:01:56.344820 19824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:56.352396 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 19824
I20260806 01:01:56.352577 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:01:56.353030 5554 external_mini_cluster.cc:1428] Setting key 8f8af1d0badb2b798da1ab8d06562181
I20260806 01:01:56.355964 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36271
--builtin_ntp_servers=127.5.108.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 01:01:56.357926 19822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:56.347342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:41255 (local address 127.5.108.190:36271)
0806 01:01:56.347639 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:56.347645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:56.348973 (+ 1328us) server_negotiation.cc:407] Connection header received
0806 01:01:56.350793 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:56.350799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:56.350889 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:56.351003 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:56.352197 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.353012 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.354807 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.355436 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.355557 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:56.356991 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:56.357001 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:56.357009 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:56.357036 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:56.357082 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:56.357101 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:56.357209 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:56.357417 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 01:01:56.357534 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260806 01:01:56.358814 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:56.347882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36271 (local address 127.5.108.130:41255)
0806 01:01:56.348814 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:56.348856 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:56.350501 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:56.351232 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:56.351240 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:56.351327 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:56.351962 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:56.351978 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.353180 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.353185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:56.354662 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:56.354674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.355633 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.355638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:56.355858 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:56.356647 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:56.356650 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:56.356703 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:56.356792 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:56.357213 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:56.357220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:56.357226 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 01:01:56.357370 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 01:01:56.357667 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":292,"threads_started":1}
I20260806 01:01:56.360317 20021 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36271
I20260806 01:01:56.360644 20021 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:56.361564 20021 heartbeater.cc:511] Master 127.5.108.190:36271 requested a full tablet report, sending...
I20260806 01:01:56.363126 19540 ts_manager.cc:194] Registered new tserver with Master: 997d537a0b8a444fac37b371ee9513ec (127.5.108.130:42533)
I20260806 01:01:56.364003 19540 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.130:41255
W20260806 01:01:56.688853 20026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:56.689095 20026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:56.689232 20026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:56.696087 20026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:56.696169 20026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:56.696357 20026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:01:56.705045 20026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:56.706194 20026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:56.707994 20026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:56.721129 20038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:56.721175 20039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:56.721215 20042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:56.721966 20026 server_base.cc:1034] running on GCE node
I20260806 01:01:56.722543 20026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:56.723555 20026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:56.724901 20026 hybrid_clock.cc:648] HybridClock initialized: now 1785978116724866 us; error 44 us; skew 500 ppm
I20260806 01:01:56.725216 20026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:56.727421 20026 webserver.cc:460] Webserver started at http://127.5.108.131:36699/ using document root <none> and password file <none>
I20260806 01:01:56.728109 20026 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:56.728269 20026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:56.728605 20026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:56.731750 20026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "9ddb9ed93242498492f935e18b7a9464"
format_stamp: "Formatted at 2026-08-06 01:01:56 on dist-test-slave-x4qh"
server_key: "552f082850e59ba8397c44a606ee23bb"
server_key_iv: "73e21af4538a7c29864e61d5ebaf9c95"
server_key_version: "encryptionkey@0"
I20260806 01:01:56.732518 20026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "9ddb9ed93242498492f935e18b7a9464"
format_stamp: "Formatted at 2026-08-06 01:01:56 on dist-test-slave-x4qh"
server_key: "552f082850e59ba8397c44a606ee23bb"
server_key_iv: "73e21af4538a7c29864e61d5ebaf9c95"
server_key_version: "encryptionkey@0"
I20260806 01:01:56.737460 20026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:01:56.740799 20053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:56.741983 20026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 01:01:56.742161 20026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "9ddb9ed93242498492f935e18b7a9464"
format_stamp: "Formatted at 2026-08-06 01:01:56 on dist-test-slave-x4qh"
server_key: "552f082850e59ba8397c44a606ee23bb"
server_key_iv: "73e21af4538a7c29864e61d5ebaf9c95"
server_key_version: "encryptionkey@0"
I20260806 01:01:56.742377 20026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:56.758121 20026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:56.758930 20026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:56.759240 20026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:56.760224 20026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:56.761619 20026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:56.761750 20026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:56.761876 20026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:56.761966 20026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:56.795588 20026 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:45785
I20260806 01:01:56.795631 20223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:45785 every 8 connection(s)
I20260806 01:01:56.796897 20026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:56.805208 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 20026
I20260806 01:01:56.805379 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:01:56.805976 5554 external_mini_cluster.cc:1428] Setting key 7f0522027acfb18213566e8c2cc40991
I20260806 01:01:56.809986 19822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:56.799411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:45149 (local address 127.5.108.190:36271)
0806 01:01:56.799628 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:56.799636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:56.800998 (+ 1362us) server_negotiation.cc:407] Connection header received
0806 01:01:56.802687 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:56.802692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:56.802764 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:56.802853 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:56.803934 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.804673 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.806941 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.807612 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.807708 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:56.809284 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:56.809288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:56.809291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:56.809308 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:56.809340 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:56.809352 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:56.809422 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:56.809624 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 01:01:56.809696 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:01:56.810959 20231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:56.799995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36271 (local address 127.5.108.131:45149)
0806 01:01:56.800858 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:56.800892 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:56.802449 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:56.803054 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:56.803062 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:56.803139 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:56.803774 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:56.803790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.804858 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.804865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:56.806782 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:01:56.806798 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.807840 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.807846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:01:56.808145 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:56.808926 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:56.808932 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:56.808990 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:56.809081 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:56.809444 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:56.809453 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:56.809458 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 01:01:56.809597 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 01:01:56.809903 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":269,"threads_started":1}
I20260806 01:01:56.812184 20224 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36271
I20260806 01:01:56.812475 20224 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:56.813186 20224 heartbeater.cc:511] Master 127.5.108.190:36271 requested a full tablet report, sending...
I20260806 01:01:56.814469 19540 ts_manager.cc:194] Registered new tserver with Master: 9ddb9ed93242498492f935e18b7a9464 (127.5.108.131:45785)
I20260806 01:01:56.815228 19540 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.131:45149
I20260806 01:01:56.821161 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 01:01:56.827131 19822 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:56.825648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.5.108.190:36271)
0806 01:01:56.825830 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:56.825835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:56.826254 (+ 419us) server_negotiation.cc:407] Connection header received
0806 01:01:56.826392 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:56.826395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:56.826450 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:56.826529 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:56.826884 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36396: BlockingRecv error: recv got EOF from 127.0.0.1:36396 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:01:56.828073 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 19624
I20260806 01:01:56.835544 20239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:56.828926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.5.108.129:34593)
0806 01:01:56.829432 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:56.829439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:56.829462 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:01:56.829515 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:56.829522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:56.829769 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:56.829888 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:56.830293 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.831305 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.832549 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.833153 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.833297 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:56.834271 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:56.834279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:56.834292 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:56.834847 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:56.834889 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:56.834907 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:56.835007 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:56.835096 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 01:01:56.835202 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":267,"threads_started":1}
I20260806 01:01:56.836705 19815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:56.917588 19820 heartbeater.cc:503] Master 127.5.108.190:36271 was elected leader, sending a full tablet report...
I20260806 01:01:56.919450 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 19624
W20260806 01:01:56.943130 19517 connection.cc:535] server connection from 127.5.108.129:49105 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 01:01:56.943436 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 19824
I20260806 01:01:56.951442 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:56.944457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45184 (local address 127.5.108.130:42533)
0806 01:01:56.945018 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:56.945025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:56.945053 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:01:56.945114 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:56.945121 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:56.945338 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:56.945469 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:56.945933 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.946982 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.948317 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:56.948908 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:56.949011 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:56.949973 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:56.949981 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:56.949995 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:56.950589 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:56.950635 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:56.950654 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:56.950741 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:56.950895 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 01:01:56.951043 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":272,"threads_started":1}
I20260806 01:01:56.952852 20013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:57.033351 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 19824
I20260806 01:01:57.058851 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 20026
I20260806 01:01:57.067034 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:57.059895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.5.108.131:45785)
0806 01:01:57.060529 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:57.060535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:57.060572 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:01:57.060640 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:57.060647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:57.060859 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:57.060986 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:57.061427 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:57.062528 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:57.063957 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:01:57.064572 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:01:57.064683 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:01:57.065668 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:57.065678 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:57.065705 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:57.066305 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:57.066357 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:57.066377 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:57.066466 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:57.066596 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 01:01:57.066722 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":323,"threads_started":1}
I20260806 01:01:57.068437 20218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:57.149428 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 20026
I20260806 01:01:57.176487 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 19492
I20260806 01:01:57.177767 19578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:57.278702 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 19492
2026-08-06T01:01:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2530 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T01:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:01:57Z Disabled control of system clock
I20260806 01:01:57.368162 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37439
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:37439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 01:01:57.701757 20255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:57.702019 20255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:57.702196 20255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:57.709190 20255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:01:57.709288 20255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:57.709396 20255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:57.709503 20255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:01:57.709561 20255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:01:57.718171 20255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:37439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:57.719345 20255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:57.721120 20255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:57.732604 20269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:57.732728 20264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:57.732686 20265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:57.733649 20255 server_base.cc:1034] running on GCE node
I20260806 01:01:57.734526 20255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:57.735929 20255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:57.737136 20255 hybrid_clock.cc:648] HybridClock initialized: now 1785978117737101 us; error 25 us; skew 500 ppm
I20260806 01:01:57.737491 20255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:57.739794 20255 webserver.cc:460] Webserver started at http://127.5.108.190:33119/ using document root <none> and password file <none>
I20260806 01:01:57.740475 20255 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:57.740595 20255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:57.740935 20255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:57.745683 20255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "12f2a961f89543e6b42a5205e85590a8"
format_stamp: "Formatted at 2026-08-06 01:01:57 on dist-test-slave-x4qh"
server_key: "b1749245671905041d01f1512f5d0c56"
server_key_iv: "62a03d1fe763ac14ca077f1ca8327d82"
server_key_version: "encryptionkey@0"
I20260806 01:01:57.746490 20255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "12f2a961f89543e6b42a5205e85590a8"
format_stamp: "Formatted at 2026-08-06 01:01:57 on dist-test-slave-x4qh"
server_key: "b1749245671905041d01f1512f5d0c56"
server_key_iv: "62a03d1fe763ac14ca077f1ca8327d82"
server_key_version: "encryptionkey@0"
I20260806 01:01:57.751569 20255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 01:01:57.754851 20278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:57.756115 20255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:57.756304 20255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "12f2a961f89543e6b42a5205e85590a8"
format_stamp: "Formatted at 2026-08-06 01:01:57 on dist-test-slave-x4qh"
server_key: "b1749245671905041d01f1512f5d0c56"
server_key_iv: "62a03d1fe763ac14ca077f1ca8327d82"
server_key_version: "encryptionkey@0"
I20260806 01:01:57.756525 20255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:57.762310 20255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:57.763041 20255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:57.763358 20255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:57.782383 20255 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:37439
I20260806 01:01:57.782425 20346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:37439 every 8 connection(s)
I20260806 01:01:57.783826 20255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:01:57.787503 20347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:01:57.791107 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 20255
I20260806 01:01:57.791399 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:01:57.792270 5554 external_mini_cluster.cc:1428] Setting key 9b5eb86f4d332f2e372bdb7b0577267c
I20260806 01:01:57.800797 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:57.794445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57558 (local address 127.5.108.190:37439)
0806 01:01:57.795539 (+ 1094us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:57.795551 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:57.795593 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:01:57.797208 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:57.797230 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:57.797649 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:57.798075 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:57.799198 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:57.799205 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:57.799217 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:57.799230 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:57.799286 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:57.799301 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:57.799560 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:57.799711 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 01:01:57.799941 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":361,"threads_started":1}
I20260806 01:01:57.800891 20347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8: Bootstrap starting.
I20260806 01:01:57.803987 20347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8: Neither blocks nor log segments found. Creating new log.
I20260806 01:01:57.805023 20347 log.cc:824] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:01:57.807691 20347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8: No bootstrap required, opened a new log
I20260806 01:01:57.815426 20347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f2a961f89543e6b42a5205e85590a8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37439 } }
I20260806 01:01:57.815798 20347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:01:57.815918 20347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12f2a961f89543e6b42a5205e85590a8, State: Initialized, Role: FOLLOWER
I20260806 01:01:57.816465 20347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f2a961f89543e6b42a5205e85590a8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37439 } }
I20260806 01:01:57.816696 20347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:01:57.816852 20347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:01:57.817046 20347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:01:57.819190 20347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f2a961f89543e6b42a5205e85590a8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37439 } }
I20260806 01:01:57.819868 20347 leader_election.cc:302] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12f2a961f89543e6b42a5205e85590a8; no voters:
I20260806 01:01:57.820488 20347 leader_election.cc:288] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:01:57.820603 20355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:01:57.821816 20355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [term 1 LEADER]: Becoming Leader. State: Replica: 12f2a961f89543e6b42a5205e85590a8, State: Running, Role: LEADER
I20260806 01:01:57.822362 20355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f2a961f89543e6b42a5205e85590a8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37439 } }
I20260806 01:01:57.822674 20347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:01:57.825207 20356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12f2a961f89543e6b42a5205e85590a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f2a961f89543e6b42a5205e85590a8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37439 } } }
I20260806 01:01:57.825187 20358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12f2a961f89543e6b42a5205e85590a8. Latest consensus state: current_term: 1 leader_uuid: "12f2a961f89543e6b42a5205e85590a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12f2a961f89543e6b42a5205e85590a8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37439 } } }
I20260806 01:01:57.825646 20356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:57.825659 20358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:01:57.828621 20363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:01:57.833870 20363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:01:57.842459 20363 catalog_manager.cc:1347] Generated new cluster ID: 409346c92fd54d70ba69306558f3dad0
I20260806 01:01:57.842592 20363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:01:57.855288 20363 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:01:57.855870 20363 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:01:57.864476 20363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12f2a961f89543e6b42a5205e85590a8: Generated new TSK 0
I20260806 01:01:57.865475 20363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:01:57.872325 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37439
--builtin_ntp_servers=127.5.108.148:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 01:01:58.214078 20378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:58.214349 20378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:58.214437 20378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:58.221367 20378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:58.221462 20378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:58.221568 20378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:01:58.231261 20378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:58.232470 20378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:58.234130 20378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:58.247061 20390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:58.247409 20393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:58.247416 20388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:58.248390 20378 server_base.cc:1034] running on GCE node
I20260806 01:01:58.248980 20378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:58.249953 20378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:58.251173 20378 hybrid_clock.cc:648] HybridClock initialized: now 1785978118251120 us; error 47 us; skew 500 ppm
I20260806 01:01:58.251500 20378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:58.253571 20378 webserver.cc:460] Webserver started at http://127.5.108.129:41285/ using document root <none> and password file <none>
I20260806 01:01:58.254221 20378 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:58.254357 20378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:58.254683 20378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:58.257601 20378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "588ba35f5e60462ba71f6801f7221841"
format_stamp: "Formatted at 2026-08-06 01:01:58 on dist-test-slave-x4qh"
server_key: "845d978913e7fb97bf3d597eb2a80bb6"
server_key_iv: "8f4b87d5308e5fa81ae2334ff35687f2"
server_key_version: "encryptionkey@0"
I20260806 01:01:58.258405 20378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "588ba35f5e60462ba71f6801f7221841"
format_stamp: "Formatted at 2026-08-06 01:01:58 on dist-test-slave-x4qh"
server_key: "845d978913e7fb97bf3d597eb2a80bb6"
server_key_iv: "8f4b87d5308e5fa81ae2334ff35687f2"
server_key_version: "encryptionkey@0"
I20260806 01:01:58.263571 20378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 01:01:58.267092 20400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:58.268494 20378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:01:58.268697 20378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "588ba35f5e60462ba71f6801f7221841"
format_stamp: "Formatted at 2026-08-06 01:01:58 on dist-test-slave-x4qh"
server_key: "845d978913e7fb97bf3d597eb2a80bb6"
server_key_iv: "8f4b87d5308e5fa81ae2334ff35687f2"
server_key_version: "encryptionkey@0"
I20260806 01:01:58.268946 20378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:58.274829 20378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:58.275684 20378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:58.275972 20378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:58.276895 20378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:58.278304 20378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:58.278456 20378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:58.278604 20378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:58.278697 20378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:58.314038 20378 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:39701
I20260806 01:01:58.314082 20586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:39701 every 8 connection(s)
I20260806 01:01:58.315336 20378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:01:58.321385 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 20378
I20260806 01:01:58.321647 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:01:58.322669 5554 external_mini_cluster.cc:1428] Setting key ae77bda339cdd1bd951773549882219c
I20260806 01:01:58.325292 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:58.318298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:35279 (local address 127.5.108.190:37439)
0806 01:01:58.318882 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:58.318892 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:58.319928 (+ 1036us) server_negotiation.cc:407] Connection header received
0806 01:01:58.322559 (+ 2631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:58.322564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:58.322620 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:58.322768 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:58.324412 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:58.324418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:58.324422 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:58.324445 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:58.324497 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:58.324512 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:58.324628 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:58.324932 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 01:01:58.324999 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":307,"threads_started":1}
I20260806 01:01:58.326462 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37439
--builtin_ntp_servers=127.5.108.148:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 01:01:58.326682 20597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:58.318596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37439 (local address 127.5.108.129:35279)
0806 01:01:58.319729 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:58.319779 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:58.322183 (+ 2404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:58.323033 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:58.323046 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:58.323203 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:58.324079 (+ 876us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:58.324082 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:58.324129 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:58.324225 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:58.324645 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:58.324655 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:58.324661 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 01:01:58.325009 (+ 348us) client_negotiation.cc:240] Negotiation successful
0806 01:01:58.325286 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":326,"threads_started":1}
I20260806 01:01:58.328644 20588 heartbeater.cc:347] Connected to a master server at 127.5.108.190:37439
I20260806 01:01:58.328946 20588 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:58.329532 20588 heartbeater.cc:511] Master 127.5.108.190:37439 requested a full tablet report, sending...
I20260806 01:01:58.331983 20304 ts_manager.cc:194] Registered new tserver with Master: 588ba35f5e60462ba71f6801f7221841 (127.5.108.129:39701)
W20260806 01:01:58.666250 20598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:58.666608 20598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:58.666790 20598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:58.674409 20598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:58.674542 20598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:58.674712 20598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:01:58.683776 20598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:58.685107 20598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:58.686890 20598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:58.699945 20607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:58.699986 20609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:58.700039 20606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:58.700951 20598 server_base.cc:1034] running on GCE node
I20260806 01:01:58.701547 20598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:58.702524 20598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:58.703750 20598 hybrid_clock.cc:648] HybridClock initialized: now 1785978118703706 us; error 32 us; skew 500 ppm
I20260806 01:01:58.704066 20598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:58.712261 20598 webserver.cc:460] Webserver started at http://127.5.108.130:43153/ using document root <none> and password file <none>
I20260806 01:01:58.712946 20598 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:58.713086 20598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:58.713436 20598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:58.716346 20598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "3d1f179f74224c3180b4c4a01c359c20"
format_stamp: "Formatted at 2026-08-06 01:01:58 on dist-test-slave-x4qh"
server_key: "5c3c8571632be05bee09f0dd751cdeef"
server_key_iv: "eca61b3dfbb9f28a9e21cc9120f53caf"
server_key_version: "encryptionkey@0"
I20260806 01:01:58.717140 20598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "3d1f179f74224c3180b4c4a01c359c20"
format_stamp: "Formatted at 2026-08-06 01:01:58 on dist-test-slave-x4qh"
server_key: "5c3c8571632be05bee09f0dd751cdeef"
server_key_iv: "eca61b3dfbb9f28a9e21cc9120f53caf"
server_key_version: "encryptionkey@0"
I20260806 01:01:58.722121 20598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:01:58.725626 20618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:58.726907 20598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 01:01:58.727073 20598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "3d1f179f74224c3180b4c4a01c359c20"
format_stamp: "Formatted at 2026-08-06 01:01:58 on dist-test-slave-x4qh"
server_key: "5c3c8571632be05bee09f0dd751cdeef"
server_key_iv: "eca61b3dfbb9f28a9e21cc9120f53caf"
server_key_version: "encryptionkey@0"
I20260806 01:01:58.727337 20598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:58.733031 20598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:58.733850 20598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:58.734134 20598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:58.735114 20598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:58.736658 20598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:58.736784 20598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:58.736927 20598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:58.737018 20598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:58.771839 20598 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:34025
I20260806 01:01:58.771927 20798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:34025 every 8 connection(s)
I20260806 01:01:58.773229 20598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:01:58.776964 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 20598
I20260806 01:01:58.777148 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:01:58.777668 5554 external_mini_cluster.cc:1428] Setting key 7616af5b4901ca71c423daf75f36f4c5
I20260806 01:01:58.780800 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37439
--builtin_ntp_servers=127.5.108.148:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 01:01:58.781977 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:58.775925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:33165 (local address 127.5.108.190:37439)
0806 01:01:58.776146 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:58.776152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:58.777531 (+ 1379us) server_negotiation.cc:407] Connection header received
0806 01:01:58.779509 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:58.779515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:58.779592 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:58.779706 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:58.781166 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:58.781171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:58.781174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:58.781190 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:58.781221 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:58.781232 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:58.781322 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:58.781583 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 01:01:58.781641 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:01:58.783022 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:58.776423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37439 (local address 127.5.108.130:33165)
0806 01:01:58.777366 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:58.777411 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:58.779193 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:58.779956 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:58.779966 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:58.780077 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:58.780775 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:58.780780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:58.780831 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:58.780946 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:58.781331 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:58.781343 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:58.781353 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:01:58.781629 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 01:01:58.781863 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":318,"threads_started":1}
I20260806 01:01:58.784327 20800 heartbeater.cc:347] Connected to a master server at 127.5.108.190:37439
I20260806 01:01:58.784629 20800 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:58.785261 20800 heartbeater.cc:511] Master 127.5.108.190:37439 requested a full tablet report, sending...
I20260806 01:01:58.786837 20304 ts_manager.cc:194] Registered new tserver with Master: 3d1f179f74224c3180b4c4a01c359c20 (127.5.108.130:34025)
W20260806 01:01:59.119451 20804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:01:59.119719 20804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:01:59.119819 20804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:01:59.127171 20804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:01:59.127267 20804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:01:59.127416 20804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:01:59.136525 20804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:01:59.137688 20804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:01:59.139353 20804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:01:59.152506 20815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:59.152583 20811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:01:59.152558 20810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:01:59.154120 20804 server_base.cc:1034] running on GCE node
I20260806 01:01:59.154637 20804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:01:59.155545 20804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:01:59.156745 20804 hybrid_clock.cc:648] HybridClock initialized: now 1785978119156688 us; error 49 us; skew 500 ppm
I20260806 01:01:59.157001 20804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:01:59.159219 20804 webserver.cc:460] Webserver started at http://127.5.108.131:37795/ using document root <none> and password file <none>
I20260806 01:01:59.159925 20804 fs_manager.cc:359] Metadata directory not provided
I20260806 01:01:59.160012 20804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:01:59.160282 20804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:01:59.163242 20804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "70542a39ce064202ab6d0d99b59e5193"
format_stamp: "Formatted at 2026-08-06 01:01:59 on dist-test-slave-x4qh"
server_key: "73bff45537a49732b08e8c83991cad35"
server_key_iv: "2a178da046fcb59eb19c635e6c8ff084"
server_key_version: "encryptionkey@0"
I20260806 01:01:59.164084 20804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "70542a39ce064202ab6d0d99b59e5193"
format_stamp: "Formatted at 2026-08-06 01:01:59 on dist-test-slave-x4qh"
server_key: "73bff45537a49732b08e8c83991cad35"
server_key_iv: "2a178da046fcb59eb19c635e6c8ff084"
server_key_version: "encryptionkey@0"
I20260806 01:01:59.170114 20804 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 01:01:59.173626 20830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:59.174906 20804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 01:01:59.175100 20804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "70542a39ce064202ab6d0d99b59e5193"
format_stamp: "Formatted at 2026-08-06 01:01:59 on dist-test-slave-x4qh"
server_key: "73bff45537a49732b08e8c83991cad35"
server_key_iv: "2a178da046fcb59eb19c635e6c8ff084"
server_key_version: "encryptionkey@0"
I20260806 01:01:59.175333 20804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:01:59.180989 20804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:01:59.181766 20804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:01:59.182045 20804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:01:59.182940 20804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:01:59.184365 20804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:01:59.184518 20804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:59.184645 20804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:01:59.184741 20804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:01:59.218453 20804 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:39957
I20260806 01:01:59.218513 20993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:39957 every 8 connection(s)
I20260806 01:01:59.219825 20804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:01:59.221933 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 20804
I20260806 01:01:59.222126 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:01:59.222679 5554 external_mini_cluster.cc:1428] Setting key 5995de7f1d8ebd189aa4a6a9b336871f
I20260806 01:01:59.228700 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:59.222474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:44737 (local address 127.5.108.190:37439)
0806 01:01:59.222734 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:59.222741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:59.224431 (+ 1690us) server_negotiation.cc:407] Connection header received
0806 01:01:59.226624 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:59.226629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:59.226695 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:59.226815 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:59.228050 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:59.228054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:59.228057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:59.228072 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:59.228104 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:59.228115 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:59.228212 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:59.228366 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 01:01:59.228414 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 01:01:59.229548 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:59.223075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37439 (local address 127.5.108.131:44737)
0806 01:01:59.224270 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0806 01:01:59.224319 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:01:59.226372 (+ 2053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:01:59.227026 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:01:59.227034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:01:59.227120 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:01:59.227775 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:01:59.227778 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:01:59.227814 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:01:59.227899 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:01:59.228206 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:01:59.228213 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:01:59.228218 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 01:01:59.228429 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 01:01:59.228683 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":357,"threads_started":1}
I20260806 01:01:59.230917 20994 heartbeater.cc:347] Connected to a master server at 127.5.108.190:37439
I20260806 01:01:59.231194 20994 heartbeater.cc:464] Registering TS with master...
I20260806 01:01:59.231938 20994 heartbeater.cc:511] Master 127.5.108.190:37439 requested a full tablet report, sending...
I20260806 01:01:59.233232 20304 ts_manager.cc:194] Registered new tserver with Master: 70542a39ce064202ab6d0d99b59e5193 (127.5.108.131:39957)
I20260806 01:01:59.238091 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 01:01:59.244037 20593 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:01:59.242605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.5.108.190:37439)
0806 01:01:59.242811 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:59.242820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:59.243214 (+ 394us) server_negotiation.cc:407] Connection header received
0806 01:01:59.243334 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:59.243337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:59.243438 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:59.243511 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:59.243782 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57564: BlockingRecv error: recv got EOF from 127.0.0.1:57564 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:01:59.244525 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 20378
I20260806 01:01:59.247893 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:59.245527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39206 (local address 127.5.108.129:39701)
0806 01:01:59.246044 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:59.246050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:59.246075 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:01:59.246128 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:59.246134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:59.246305 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:59.246418 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:59.247230 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:59.247235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:59.247246 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:59.247265 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:59.247304 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:59.247317 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:59.247481 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:59.247580 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 01:01:59.247663 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":263,"threads_started":1}
I20260806 01:01:59.249020 20581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:59.330493 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 20378
I20260806 01:01:59.354859 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 20598
I20260806 01:01:59.358497 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:59.355872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.5.108.130:34025)
0806 01:01:59.356435 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:59.356442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:59.356470 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:01:59.356537 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:59.356543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:59.356772 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:59.356905 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:59.357853 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:59.357859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:59.357871 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:59.357890 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:59.357933 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:59.357947 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:59.358049 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:59.358175 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 01:01:59.358260 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":277,"threads_started":1}
I20260806 01:01:59.359786 20793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:59.441107 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 20598
I20260806 01:01:59.463946 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 20804
I20260806 01:01:59.468128 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:01:59.465026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60796 (local address 127.5.108.131:39957)
0806 01:01:59.465683 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 01:01:59.465690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:01:59.465721 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:01:59.465782 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:01:59.465788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:01:59.465993 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:01:59.466165 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:01:59.467304 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:01:59.467310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:01:59.467320 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:01:59.467336 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:01:59.467425 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:01:59.467446 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:01:59.467639 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 01:01:59.467724 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 01:01:59.467824 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":331,"threads_started":1}
I20260806 01:01:59.469323 20986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:59.551872 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 20804
I20260806 01:01:59.579135 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 20255
I20260806 01:01:59.580385 20342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:01:59.676599 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 20255
2026-08-06T01:01:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2398 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:01:59 dist-test-slave-x4qh krb5kdc[21019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:01:59 dist-test-slave-x4qh krb5kdc[21019](info): set up 2 sockets
Aug 06 01:01:59 dist-test-slave-x4qh krb5kdc[21019](info): commencing operation
krb5kdc: starting...
W20260806 01:02:00.952530 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:02:01 dist-test-slave-x4qh krb5kdc[21019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:02:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:02:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:02:01.218132 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:43309
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:43309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:02:01.557209 21039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:02:01.557547 21039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:02:01.557713 21039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:02:01.565456 21039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:02:01.565606 21039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:02:01.565724 21039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:02:01.565832 21039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:02:01.565899 21039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:02:01.575865 21039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:43309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:43309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:02:01.577184 21039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:02:01.578581 21039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:02:01.590575 21045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:01.590476 21046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:01.590436 21048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:02:01.591691 21039 server_base.cc:1034] running on GCE node
I20260806 01:02:01.592365 21039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:02:01.593703 21039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:02:01.594913 21039 hybrid_clock.cc:648] HybridClock initialized: now 1785978121594872 us; error 33 us; skew 500 ppm
I20260806 01:02:01.595223 21039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:02:01 dist-test-slave-x4qh krb5kdc[21019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978121, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:02:01.598205 21039 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:02:01.599624 21039 webserver.cc:460] Webserver started at http://127.5.108.190:34679/ using document root <none> and password file <none>
I20260806 01:02:01.600241 21039 fs_manager.cc:359] Metadata directory not provided
I20260806 01:02:01.600345 21039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:02:01.600579 21039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:02:01.603434 21039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "55bebab39cb14a7bb2b26676aee52cb4"
format_stamp: "Formatted at 2026-08-06 01:02:01 on dist-test-slave-x4qh"
server_key: "65c205bb46a224c31fbdd41681b7ef12"
server_key_iv: "f4f8637d0f3f5e3c824a6dfd687d1d01"
server_key_version: "encryptionkey@0"
I20260806 01:02:01.604182 21039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "55bebab39cb14a7bb2b26676aee52cb4"
format_stamp: "Formatted at 2026-08-06 01:02:01 on dist-test-slave-x4qh"
server_key: "65c205bb46a224c31fbdd41681b7ef12"
server_key_iv: "f4f8637d0f3f5e3c824a6dfd687d1d01"
server_key_version: "encryptionkey@0"
I20260806 01:02:01.609061 21039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:02:01.612248 21061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:01.613572 21039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:02:01.613703 21039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "55bebab39cb14a7bb2b26676aee52cb4"
format_stamp: "Formatted at 2026-08-06 01:02:01 on dist-test-slave-x4qh"
server_key: "65c205bb46a224c31fbdd41681b7ef12"
server_key_iv: "f4f8637d0f3f5e3c824a6dfd687d1d01"
server_key_version: "encryptionkey@0"
I20260806 01:02:01.613855 21039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:02:01.648464 21039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:02:01.652767 21039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:02:01.652997 21039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:02:01.671895 21039 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:43309
I20260806 01:02:01.671993 21131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:43309 every 8 connection(s)
I20260806 01:02:01.673322 21039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:02:01.674038 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 21039
I20260806 01:02:01.674319 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:02:01.675103 5554 external_mini_cluster.cc:1428] Setting key 4fe82f916c880ee93597fe3cab9dc538
I20260806 01:02:01.677698 21132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:02:01 dist-test-slave-x4qh krb5kdc[21019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:02:01.690878 21132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4: Bootstrap starting.
I20260806 01:02:01.695062 21132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4: Neither blocks nor log segments found. Creating new log.
I20260806 01:02:01.696333 21132 log.cc:824] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4: Log is configured to *not* fsync() on all Append() calls
I20260806 01:02:01.699712 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:01.677196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.5.108.190:43309)
0806 01:02:01.678047 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:01.678058 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:01.678103 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:02:01.679902 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:01.679923 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:01.680292 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:01.680672 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:01.682478 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:01.683920 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:01.685683 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:01.686320 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:01.686450 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:01.691153 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:01.691166 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:01.691184 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:01.691226 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:01.695136 (+ 3910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:01.695886 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:01.695894 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:01.695903 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:01.696134 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:01.696515 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:01.696521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:01.696524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:01.697225 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:01.697460 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:01.698019 (+ 559us) server_negotiation.cc:299] Negotiation successful
0806 01:02:01.698410 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":420,"threads_started":1}
I20260806 01:02:01.699755 21132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4: No bootstrap required, opened a new log
I20260806 01:02:01.707026 21132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55bebab39cb14a7bb2b26676aee52cb4" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 43309 } }
I20260806 01:02:01.707357 21132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:02:01.707525 21132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55bebab39cb14a7bb2b26676aee52cb4, State: Initialized, Role: FOLLOWER
I20260806 01:02:01.707986 21132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55bebab39cb14a7bb2b26676aee52cb4" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 43309 } }
I20260806 01:02:01.708191 21132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:02:01.708297 21132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:02:01.708423 21132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:02:01.710412 21132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55bebab39cb14a7bb2b26676aee52cb4" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 43309 } }
I20260806 01:02:01.710826 21132 leader_election.cc:302] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55bebab39cb14a7bb2b26676aee52cb4; no voters:
I20260806 01:02:01.711539 21132 leader_election.cc:288] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:02:01.711721 21139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:02:01.712491 21139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [term 1 LEADER]: Becoming Leader. State: Replica: 55bebab39cb14a7bb2b26676aee52cb4, State: Running, Role: LEADER
I20260806 01:02:01.712986 21139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55bebab39cb14a7bb2b26676aee52cb4" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 43309 } }
I20260806 01:02:01.713665 21132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:02:01.716046 21142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55bebab39cb14a7bb2b26676aee52cb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55bebab39cb14a7bb2b26676aee52cb4" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 43309 } } }
I20260806 01:02:01.716437 21142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [sys.catalog]: This master's current role is: LEADER
I20260806 01:02:01.716166 21143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55bebab39cb14a7bb2b26676aee52cb4. Latest consensus state: current_term: 1 leader_uuid: "55bebab39cb14a7bb2b26676aee52cb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55bebab39cb14a7bb2b26676aee52cb4" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 43309 } } }
I20260806 01:02:01.716797 21143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4 [sys.catalog]: This master's current role is: LEADER
I20260806 01:02:01.720436 21151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:02:01.725726 21151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:02:01.734577 21151 catalog_manager.cc:1347] Generated new cluster ID: fbf7dae5d2ff4a5d9ecca661e103b4af
I20260806 01:02:01.734684 21151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:02:01.746768 21151 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:02:01.748582 21151 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:02:01.763681 21151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4: Generated new TSK 0
I20260806 01:02:01.764508 21151 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:02:01.839501 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:43309
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:02:02.170179 21172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:02:02.170429 21172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:02:02.170595 21172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:02:02.179631 21172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:02:02.179785 21172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:02:02.179988 21172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:02:02.189606 21172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:43309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:02:02.190735 21172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:02:02.192047 21172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:02:02.205549 21179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:02.205577 21181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:02.206135 21178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:02:02.206471 21172 server_base.cc:1034] running on GCE node
I20260806 01:02:02.207151 21172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:02:02.208108 21172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:02:02.209318 21172 hybrid_clock.cc:648] HybridClock initialized: now 1785978122209286 us; error 29 us; skew 500 ppm
I20260806 01:02:02.209642 21172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:02:02 dist-test-slave-x4qh krb5kdc[21019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978122, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:02:02.212805 21172 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:02:02.214283 21172 webserver.cc:460] Webserver started at http://127.5.108.129:42905/ using document root <none> and password file <none>
I20260806 01:02:02.215078 21172 fs_manager.cc:359] Metadata directory not provided
I20260806 01:02:02.215220 21172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:02:02.215616 21172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:02:02.218631 21172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "75c357abc48947f9a98a9b0c868fa630"
format_stamp: "Formatted at 2026-08-06 01:02:02 on dist-test-slave-x4qh"
server_key: "8923bd40454b0989c943806f1749f89b"
server_key_iv: "40527dca774f358a6aa3e550a07125a6"
server_key_version: "encryptionkey@0"
I20260806 01:02:02.219451 21172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "75c357abc48947f9a98a9b0c868fa630"
format_stamp: "Formatted at 2026-08-06 01:02:02 on dist-test-slave-x4qh"
server_key: "8923bd40454b0989c943806f1749f89b"
server_key_iv: "40527dca774f358a6aa3e550a07125a6"
server_key_version: "encryptionkey@0"
I20260806 01:02:02.224500 21172 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:02:02.227806 21189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:02.229159 21172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:02:02.229302 21172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "75c357abc48947f9a98a9b0c868fa630"
format_stamp: "Formatted at 2026-08-06 01:02:02 on dist-test-slave-x4qh"
server_key: "8923bd40454b0989c943806f1749f89b"
server_key_iv: "40527dca774f358a6aa3e550a07125a6"
server_key_version: "encryptionkey@0"
I20260806 01:02:02.229470 21172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:02:02.240818 21172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:02:02.244593 21172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:02:02.244827 21172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:02:02.245669 21172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:02:02.246834 21172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:02:02.246905 21172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:02.246984 21172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:02:02.247027 21172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:02.279038 21172 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:35875
I20260806 01:02:02.279083 21356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:35875 every 8 connection(s)
I20260806 01:02:02.280431 21172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:02:02.285024 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 21172
I20260806 01:02:02.285275 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:02:02.286051 5554 external_mini_cluster.cc:1428] Setting key a309976a6f6123a3e369aa453d63d2b1
Aug 06 01:02:02 dist-test-slave-x4qh krb5kdc[21019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978122, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:02:02.302510 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:02.283099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:46677 (local address 127.5.108.190:43309)
0806 01:02:02.283588 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:02.283592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:02.284537 (+ 945us) server_negotiation.cc:407] Connection header received
0806 01:02:02.286480 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:02.286486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:02.286577 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:02.286699 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:02.288445 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:02.289158 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:02.290744 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:02.291236 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:02.291442 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:02.295212 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:02.295222 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:02.295229 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:02.295276 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:02.298995 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:02.299880 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:02.299888 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:02.299893 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:02.300049 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:02.300477 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:02.300485 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:02.300490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:02.300883 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:02.301038 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:02.301790 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 01:02:02.302057 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":248,"threads_started":1}
I20260806 01:02:02.303141 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:02.283689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:43309 (local address 127.5.108.129:46677)
0806 01:02:02.284386 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:02.284429 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:02.286220 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:02.286935 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:02.286944 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:02.287693 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:02:02.288278 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:02.288295 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:02.289328 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:02.289333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:02.290566 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:02.290583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:02.291546 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:02.291555 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:02.291833 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:02.292535 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:02:02.292557 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:02:02.294972 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:02:02.299160 (+ 4188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:02.299168 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:02.299180 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:02.299741 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:02.300161 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:02.300164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:02.300166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:02.300357 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:02.301004 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:02:02.301009 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:02:02.301342 (+ 333us) client_negotiation.cc:769] Sending connection context
0806 01:02:02.301570 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 01:02:02.302021 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":280,"threads_started":1}
I20260806 01:02:02.304872 21357 heartbeater.cc:347] Connected to a master server at 127.5.108.190:43309
I20260806 01:02:02.305195 21357 heartbeater.cc:464] Registering TS with master...
I20260806 01:02:02.305933 21357 heartbeater.cc:511] Master 127.5.108.190:43309 requested a full tablet report, sending...
I20260806 01:02:02.308341 21085 ts_manager.cc:194] Registered new tserver with Master: 75c357abc48947f9a98a9b0c868fa630 (127.5.108.129:35875)
I20260806 01:02:02.310639 21085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:46677
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:02:02.354434 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:43309
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:02:02.692425 21365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:02:02.692693 21365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:02:02.692791 21365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:02:02.700165 21365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:02:02.700270 21365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:02:02.700482 21365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:02:02.710650 21365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:43309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:02:02.712074 21365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:02:02.713433 21365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:02:02.727145 21372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:02.727094 21376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:02.727104 21373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:02:02.728402 21365 server_base.cc:1034] running on GCE node
I20260806 01:02:02.728971 21365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:02:02.729872 21365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:02:02.731099 21365 hybrid_clock.cc:648] HybridClock initialized: now 1785978122731054 us; error 59 us; skew 500 ppm
I20260806 01:02:02.731464 21365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:02:02 dist-test-slave-x4qh krb5kdc[21019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978122, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:02:02.734720 21365 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:02:02.736311 21365 webserver.cc:460] Webserver started at http://127.5.108.130:33339/ using document root <none> and password file <none>
I20260806 01:02:02.737010 21365 fs_manager.cc:359] Metadata directory not provided
I20260806 01:02:02.737164 21365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:02:02.737488 21365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:02:02.740469 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "2ee2da566691468489827d1f7227ee7e"
format_stamp: "Formatted at 2026-08-06 01:02:02 on dist-test-slave-x4qh"
server_key: "3fee992188935f341ec65dcd2934419d"
server_key_iv: "9e96778b7d73d3fb1e55f22408086bb5"
server_key_version: "encryptionkey@0"
I20260806 01:02:02.741266 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "2ee2da566691468489827d1f7227ee7e"
format_stamp: "Formatted at 2026-08-06 01:02:02 on dist-test-slave-x4qh"
server_key: "3fee992188935f341ec65dcd2934419d"
server_key_iv: "9e96778b7d73d3fb1e55f22408086bb5"
server_key_version: "encryptionkey@0"
I20260806 01:02:02.746398 21365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:02:02.749891 21391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:02.751161 21365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:02:02.751350 21365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "2ee2da566691468489827d1f7227ee7e"
format_stamp: "Formatted at 2026-08-06 01:02:02 on dist-test-slave-x4qh"
server_key: "3fee992188935f341ec65dcd2934419d"
server_key_iv: "9e96778b7d73d3fb1e55f22408086bb5"
server_key_version: "encryptionkey@0"
I20260806 01:02:02.751622 21365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:02:02.783943 21365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:02:02.788218 21365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:02:02.788520 21365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:02:02.789486 21365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:02:02.790829 21365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:02:02.790940 21365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:02.791103 21365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:02:02.791188 21365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:02.825157 21365 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:41029
I20260806 01:02:02.825205 21562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:41029 every 8 connection(s)
I20260806 01:02:02.826490 21365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:02:02.833726 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 21365
I20260806 01:02:02.833947 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:02:02.834508 5554 external_mini_cluster.cc:1428] Setting key 15c4b30ba2b9751e34ec77e7031e6bb7
Aug 06 01:02:02 dist-test-slave-x4qh krb5kdc[21019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978122, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:02:02.848390 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:02.829128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:48359 (local address 127.5.108.190:43309)
0806 01:02:02.829598 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:02.829603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:02.830375 (+ 772us) server_negotiation.cc:407] Connection header received
0806 01:02:02.832291 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:02.832297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:02.832386 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:02.832511 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:02.834349 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:02.835521 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:02.837586 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:02.838045 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:02.838156 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:02.842412 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:02.842418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:02.842421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:02.842455 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:02.845181 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:02.845933 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:02.845939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:02.845942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:02.846080 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:02.846542 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:02.846548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:02.846551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:02.846825 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:02.847004 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:02.847755 (+ 751us) server_negotiation.cc:299] Negotiation successful
0806 01:02:02.847984 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":262,"threads_started":1}
I20260806 01:02:02.849344 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:02.829549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:43309 (local address 127.5.108.130:48359)
0806 01:02:02.830216 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:02.830259 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:02.832009 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:02.832751 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:02.832759 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:02.833290 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:02:02.834136 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:02.834161 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:02.835719 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:02.835723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:02.837430 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:02.837448 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:02.838289 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:02.838298 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:02.838633 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:02.839454 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:02:02.839495 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:02:02.842166 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:02:02.845335 (+ 3169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:02.845345 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:02.845361 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:02.845792 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:02.846206 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:02.846212 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:02.846215 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:02.846421 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:02.846967 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:02:02.846975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:02:02.847323 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 01:02:02.847687 (+ 364us) client_negotiation.cc:240] Negotiation successful
0806 01:02:02.848209 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":287,"threads_started":1}
I20260806 01:02:02.850904 21563 heartbeater.cc:347] Connected to a master server at 127.5.108.190:43309
I20260806 01:02:02.851248 21563 heartbeater.cc:464] Registering TS with master...
I20260806 01:02:02.852140 21563 heartbeater.cc:511] Master 127.5.108.190:43309 requested a full tablet report, sending...
I20260806 01:02:02.853864 21085 ts_manager.cc:194] Registered new tserver with Master: 2ee2da566691468489827d1f7227ee7e (127.5.108.130:41029)
I20260806 01:02:02.855194 21085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:48359
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:02:02.900144 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:43309
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:02:03.238663 21571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:02:03.238945 21571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:02:03.239063 21571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:02:03.246237 21571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:02:03.246351 21571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:02:03.246483 21571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:02:03.256575 21571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:43309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:02:03.257766 21571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:02:03.259045 21571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:02:03.272636 21579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:03.272686 21581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:03.272722 21578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:02:03.273515 21571 server_base.cc:1034] running on GCE node
I20260806 01:02:03.274145 21571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:02:03.275012 21571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:02:03.276230 21571 hybrid_clock.cc:648] HybridClock initialized: now 1785978123276157 us; error 39 us; skew 500 ppm
I20260806 01:02:03.276517 21571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:02:03 dist-test-slave-x4qh krb5kdc[21019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978123, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:02:03.280121 21571 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:02:03.284718 21571 webserver.cc:460] Webserver started at http://127.5.108.131:44599/ using document root <none> and password file <none>
I20260806 01:02:03.285450 21571 fs_manager.cc:359] Metadata directory not provided
I20260806 01:02:03.285620 21571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:02:03.285956 21571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:02:03.288962 21571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "1951a3f81efa467393b5e6852ea28a85"
format_stamp: "Formatted at 2026-08-06 01:02:03 on dist-test-slave-x4qh"
server_key: "e0c6e2b96f7507f6865a928141606dfd"
server_key_iv: "c28c340c76da3836c339b93caa6edc45"
server_key_version: "encryptionkey@0"
I20260806 01:02:03.289901 21571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "1951a3f81efa467393b5e6852ea28a85"
format_stamp: "Formatted at 2026-08-06 01:02:03 on dist-test-slave-x4qh"
server_key: "e0c6e2b96f7507f6865a928141606dfd"
server_key_iv: "c28c340c76da3836c339b93caa6edc45"
server_key_version: "encryptionkey@0"
I20260806 01:02:03.295147 21571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260806 01:02:03.298734 21589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:03.300065 21571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:02:03.300253 21571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "1951a3f81efa467393b5e6852ea28a85"
format_stamp: "Formatted at 2026-08-06 01:02:03 on dist-test-slave-x4qh"
server_key: "e0c6e2b96f7507f6865a928141606dfd"
server_key_iv: "c28c340c76da3836c339b93caa6edc45"
server_key_version: "encryptionkey@0"
I20260806 01:02:03.300464 21571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:02:03.313656 21357 heartbeater.cc:503] Master 127.5.108.190:43309 was elected leader, sending a full tablet report...
I20260806 01:02:03.329210 21571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:02:03.333613 21571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:02:03.333935 21571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:02:03.334939 21571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:02:03.336440 21571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:02:03.336519 21571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:03.336623 21571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:02:03.336671 21571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:03.372869 21571 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:44613
I20260806 01:02:03.372912 21765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:44613 every 8 connection(s)
I20260806 01:02:03.374267 21571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:02:03.378476 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 21571
I20260806 01:02:03.378747 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:02:03.379407 5554 external_mini_cluster.cc:1428] Setting key caecc893455f2ddcac70b8ab6b4a47d7
Aug 06 01:02:03 dist-test-slave-x4qh krb5kdc[21019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978123, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:02:03.395757 21769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.376995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:58953 (local address 127.5.108.190:43309)
0806 01:02:03.377560 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.377564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.378452 (+ 888us) server_negotiation.cc:407] Connection header received
0806 01:02:03.381099 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.381103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.381167 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.381263 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:03.383414 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.384209 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.385847 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.386229 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.386325 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.389801 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:03.389806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:03.389809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:03.389839 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:03.392487 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:03.393222 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:03.393226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:03.393228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:03.393352 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:03.393830 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:03.393838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:03.393841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:03.394198 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:03.394387 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.395134 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.395426 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":291,"threads_started":1}
I20260806 01:02:03.396552 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.377433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:43309 (local address 127.5.108.131:58953)
0806 01:02:03.378266 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:03.378333 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:03.380814 (+ 2481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:03.381574 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:03.381582 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:03.382411 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:02:03.383220 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.383240 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.384378 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.384382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.385720 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.385733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.386425 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.386430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.386632 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.387261 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:02:03.387284 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:02:03.389625 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:02:03.392611 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:03.392619 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:03.392631 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:03.393076 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:03.393479 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:03.393482 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:03.393485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:03.393681 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:03.394339 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:02:03.394345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:02:03.394718 (+ 373us) client_negotiation.cc:769] Sending connection context
0806 01:02:03.394963 (+ 245us) client_negotiation.cc:240] Negotiation successful
0806 01:02:03.395496 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":300,"threads_started":1}
I20260806 01:02:03.397862 21766 heartbeater.cc:347] Connected to a master server at 127.5.108.190:43309
I20260806 01:02:03.398150 21766 heartbeater.cc:464] Registering TS with master...
I20260806 01:02:03.398840 21766 heartbeater.cc:511] Master 127.5.108.190:43309 requested a full tablet report, sending...
I20260806 01:02:03.400295 21085 ts_manager.cc:194] Registered new tserver with Master: 1951a3f81efa467393b5e6852ea28a85 (127.5.108.131:44613)
I20260806 01:02:03.401219 21085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:58953
I20260806 01:02:03.407305 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:02:03.424345 21769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.411961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.5.108.190:43309)
0806 01:02:03.412120 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.412124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.412597 (+ 473us) server_negotiation.cc:407] Connection header received
0806 01:02:03.412732 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.412746 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.412806 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.412882 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:03.414462 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.415182 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.416396 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.416823 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.416925 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.419280 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:03.419284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:03.419287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:03.419315 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:03.422087 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:03.422693 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:03.422697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:03.422699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:03.422811 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:03.423138 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:03.423141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:03.423143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:03.423454 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:03.423634 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.423941 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.424112 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:02:03.427986 21085 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:02:03.434900 21085 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:02:03.468766 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.447011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58258 (local address 127.5.108.129:35875)
0806 01:02:03.457223 (+ 10212us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.457231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.458165 (+ 934us) server_negotiation.cc:407] Connection header received
0806 01:02:03.458287 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.458298 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.458587 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.459793 (+ 1206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:03.460084 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.462094 (+ 2010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.465849 (+ 3755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.467315 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.467481 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.467726 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.468223 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.468397 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9975,"thread_start_us":9835,"threads_started":1}
I20260806 01:02:03.469789 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.446861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35875 (local address 127.0.0.1:58258)
0806 01:02:03.457662 (+ 10801us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:03.457722 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:03.457938 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:03.458869 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:03.458880 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:03.458920 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:03.459511 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.459528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.462956 (+ 3428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.462965 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.465693 (+ 2728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.465704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.467558 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.467564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.467916 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.467988 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 01:02:03.468127 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 01:02:03.469300 (+ 1173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10475,"mutex_wait_us":87,"thread_start_us":17457,"threads_started":1}
I20260806 01:02:03.472496 21792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.447751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:44613 (local address 127.0.0.1:41660)
0806 01:02:03.464055 (+ 16304us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:03.464073 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:03.464220 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:03.464831 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:03.464836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:03.464861 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:03.465269 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.465282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.467161 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.467167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.469428 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.469449 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.471641 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.471650 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.471837 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.471869 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:02:03.471972 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 01:02:03.472086 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16162,"mutex_wait_us":20,"thread_start_us":9453,"threads_started":1}
I20260806 01:02:03.472568 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.448012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.5.108.131:44613)
0806 01:02:03.462821 (+ 14809us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.462832 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.464196 (+ 1364us) server_negotiation.cc:407] Connection header received
0806 01:02:03.464416 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.464424 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.464633 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.464814 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:03.465475 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.467028 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.469603 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.471413 (+ 1810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.471543 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.471759 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.472031 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.472214 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14575,"thread_start_us":15366,"threads_started":1}
I20260806 01:02:03.473043 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.458262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38426 (local address 127.5.108.130:41029)
0806 01:02:03.460386 (+ 2124us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.460394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.464578 (+ 4184us) server_negotiation.cc:407] Connection header received
0806 01:02:03.464688 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.464695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.464912 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.465098 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:03.465994 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.467519 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.469985 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.471420 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.471544 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.471729 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.472598 (+ 869us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.472801 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1956,"thread_start_us":1817,"threads_started":1}
I20260806 01:02:03.473409 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.460443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:41029 (local address 127.0.0.1:38426)
0806 01:02:03.464298 (+ 3855us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:03.464317 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:03.464452 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:03.465296 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:03.465299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:03.465322 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:03.465818 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.465831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.467714 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.467719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.469789 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.469806 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.471894 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.471898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.472073 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.472106 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:02:03.472198 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 01:02:03.472321 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3672,"mutex_wait_us":2,"thread_start_us":3247,"threads_started":1}
I20260806 01:02:03.473590 21258 tablet_service.cc:1467] Processing CreateTablet for tablet 084d5d41f8e5434fbb9be2ea9f27c25c (DEFAULT_TABLE table=test-table [id=91b465ac54df48e6ac4e49553fad770e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:02:03.476459 21258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084d5d41f8e5434fbb9be2ea9f27c25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:02:03.476619 21459 tablet_service.cc:1467] Processing CreateTablet for tablet 084d5d41f8e5434fbb9be2ea9f27c25c (DEFAULT_TABLE table=test-table [id=91b465ac54df48e6ac4e49553fad770e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:02:03.476795 21668 tablet_service.cc:1467] Processing CreateTablet for tablet 084d5d41f8e5434fbb9be2ea9f27c25c (DEFAULT_TABLE table=test-table [id=91b465ac54df48e6ac4e49553fad770e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:02:03.477749 21459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084d5d41f8e5434fbb9be2ea9f27c25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:02:03.478117 21668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084d5d41f8e5434fbb9be2ea9f27c25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:02:03.488521 21793 tablet_bootstrap.cc:492] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: Bootstrap starting.
I20260806 01:02:03.490933 21794 tablet_bootstrap.cc:492] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: Bootstrap starting.
I20260806 01:02:03.492045 21793 tablet_bootstrap.cc:654] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: Neither blocks nor log segments found. Creating new log.
I20260806 01:02:03.492439 21795 tablet_bootstrap.cc:492] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: Bootstrap starting.
I20260806 01:02:03.493436 21793 log.cc:824] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:02:03.494558 21794 tablet_bootstrap.cc:654] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: Neither blocks nor log segments found. Creating new log.
I20260806 01:02:03.495719 21795 tablet_bootstrap.cc:654] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: Neither blocks nor log segments found. Creating new log.
I20260806 01:02:03.495901 21794 log.cc:824] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: Log is configured to *not* fsync() on all Append() calls
I20260806 01:02:03.496909 21793 tablet_bootstrap.cc:492] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: No bootstrap required, opened a new log
I20260806 01:02:03.496876 21795 log.cc:824] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: Log is configured to *not* fsync() on all Append() calls
I20260806 01:02:03.497184 21793 ts_tablet_manager.cc:1397] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 01:02:03.498951 21794 tablet_bootstrap.cc:492] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: No bootstrap required, opened a new log
I20260806 01:02:03.499208 21794 ts_tablet_manager.cc:1397] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 01:02:03.499248 21795 tablet_bootstrap.cc:492] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: No bootstrap required, opened a new log
I20260806 01:02:03.499521 21795 ts_tablet_manager.cc:1397] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260806 01:02:03.507176 21795 raft_consensus.cc:348] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } }
I20260806 01:02:03.507803 21795 raft_consensus.cc:374] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:02:03.507964 21795 raft_consensus.cc:729] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1951a3f81efa467393b5e6852ea28a85, State: Initialized, Role: FOLLOWER
I20260806 01:02:03.508785 21795 consensus_queue.cc:260] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } }
I20260806 01:02:03.509027 21793 raft_consensus.cc:348] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } }
I20260806 01:02:03.509735 21793 raft_consensus.cc:374] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:02:03.509936 21793 raft_consensus.cc:729] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ee2da566691468489827d1f7227ee7e, State: Initialized, Role: FOLLOWER
I20260806 01:02:03.509721 21794 raft_consensus.cc:348] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } }
I20260806 01:02:03.510135 21794 raft_consensus.cc:374] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:02:03.510237 21794 raft_consensus.cc:729] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c357abc48947f9a98a9b0c868fa630, State: Initialized, Role: FOLLOWER
I20260806 01:02:03.510895 21794 consensus_queue.cc:260] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } }
I20260806 01:02:03.511474 21793 consensus_queue.cc:260] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } }
I20260806 01:02:03.516105 21794 ts_tablet_manager.cc:1428] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
I20260806 01:02:03.517954 21766 heartbeater.cc:503] Master 127.5.108.190:43309 was elected leader, sending a full tablet report...
I20260806 01:02:03.518343 21563 heartbeater.cc:503] Master 127.5.108.190:43309 was elected leader, sending a full tablet report...
I20260806 01:02:03.518668 21795 ts_tablet_manager.cc:1428] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: Time spent starting tablet: real 0.019s user 0.010s sys 0.008s
I20260806 01:02:03.520092 21793 ts_tablet_manager.cc:1428] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: Time spent starting tablet: real 0.023s user 0.007s sys 0.010s
W20260806 01:02:03.532765 21358 tablet.cc:2369] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:02:03.578715 21564 tablet.cc:2369] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:02:03.612732 21804 raft_consensus.cc:482] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:02:03.612999 21804 raft_consensus.cc:504] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } }
I20260806 01:02:03.614723 21804 leader_election.cc:288] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75c357abc48947f9a98a9b0c868fa630 (127.5.108.129:35875), 1951a3f81efa467393b5e6852ea28a85 (127.5.108.131:44613)
I20260806 01:02:03.621598 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.615056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:35449 (local address 127.5.108.129:35875)
0806 01:02:03.615319 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.615328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.616135 (+ 807us) server_negotiation.cc:407] Connection header received
0806 01:02:03.616312 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.616316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.616389 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.616476 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:03.617089 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.617859 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.619793 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.620648 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.620834 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.620929 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.621213 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.621338 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:02:03.621770 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.615417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:39377 (local address 127.5.108.131:44613)
0806 01:02:03.615608 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.615615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.616203 (+ 588us) server_negotiation.cc:407] Connection header received
0806 01:02:03.616417 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.616422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.616481 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.616590 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:03.617291 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.618207 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.619992 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.620829 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.620998 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.621078 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.621387 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.621502 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 01:02:03.621830 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.614935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35875 (local address 127.5.108.130:35449)
0806 01:02:03.616023 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:03.616038 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:03.616156 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:03.616508 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:03.616512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:03.616536 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:03.616931 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.616940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.617986 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.617993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.619636 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.619649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.620855 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.620862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.621037 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.621080 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 01:02:03.621193 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 01:02:03.621387 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"mutex_wait_us":48,"thread_start_us":714,"threads_started":1}
I20260806 01:02:03.622352 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.615181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:44613 (local address 127.5.108.130:39377)
0806 01:02:03.616081 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:03.616129 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:03.616266 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:03.616611 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:03.616615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:03.616654 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:03.617136 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.617146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.618335 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.618340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.619845 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:03.619857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.621028 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.621034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:03.621228 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.621251 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 01:02:03.621361 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 01:02:03.621477 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":26,"thread_start_us":588,"threads_started":1}
I20260806 01:02:03.622985 21288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084d5d41f8e5434fbb9be2ea9f27c25c" candidate_uuid: "2ee2da566691468489827d1f7227ee7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75c357abc48947f9a98a9b0c868fa630" is_pre_election: true
I20260806 01:02:03.623524 21288 raft_consensus.cc:2454] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ee2da566691468489827d1f7227ee7e in term 0.
I20260806 01:02:03.623485 21701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084d5d41f8e5434fbb9be2ea9f27c25c" candidate_uuid: "2ee2da566691468489827d1f7227ee7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1951a3f81efa467393b5e6852ea28a85" is_pre_election: true
I20260806 01:02:03.624015 21701 raft_consensus.cc:2454] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ee2da566691468489827d1f7227ee7e in term 0.
I20260806 01:02:03.624197 21394 leader_election.cc:302] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ee2da566691468489827d1f7227ee7e, 75c357abc48947f9a98a9b0c868fa630; no voters:
I20260806 01:02:03.624635 21804 raft_consensus.cc:2781] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:02:03.624789 21804 raft_consensus.cc:482] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:02:03.624894 21804 raft_consensus.cc:3037] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 0 FOLLOWER]: Advancing to term 1
W20260806 01:02:03.626154 21767 tablet.cc:2369] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:02:03.627200 21804 raft_consensus.cc:504] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } }
I20260806 01:02:03.627887 21804 leader_election.cc:288] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [CANDIDATE]: Term 1 election: Requested vote from peers 75c357abc48947f9a98a9b0c868fa630 (127.5.108.129:35875), 1951a3f81efa467393b5e6852ea28a85 (127.5.108.131:44613)
I20260806 01:02:03.628321 21288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084d5d41f8e5434fbb9be2ea9f27c25c" candidate_uuid: "2ee2da566691468489827d1f7227ee7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75c357abc48947f9a98a9b0c868fa630"
I20260806 01:02:03.628350 21701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084d5d41f8e5434fbb9be2ea9f27c25c" candidate_uuid: "2ee2da566691468489827d1f7227ee7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1951a3f81efa467393b5e6852ea28a85"
I20260806 01:02:03.628520 21288 raft_consensus.cc:3037] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:02:03.628520 21701 raft_consensus.cc:3037] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:02:03.630666 21701 raft_consensus.cc:2454] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ee2da566691468489827d1f7227ee7e in term 1.
I20260806 01:02:03.630784 21288 raft_consensus.cc:2454] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ee2da566691468489827d1f7227ee7e in term 1.
I20260806 01:02:03.631114 21396 leader_election.cc:302] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1951a3f81efa467393b5e6852ea28a85, 2ee2da566691468489827d1f7227ee7e; no voters:
I20260806 01:02:03.631455 21804 raft_consensus.cc:2781] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:02:03.631915 21804 raft_consensus.cc:686] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 1 LEADER]: Becoming Leader. State: Replica: 2ee2da566691468489827d1f7227ee7e, State: Running, Role: LEADER
I20260806 01:02:03.632411 21804 consensus_queue.cc:237] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } }
I20260806 01:02:03.637306 21084 catalog_manager.cc:5495] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ee2da566691468489827d1f7227ee7e (127.5.108.130). New cstate: current_term: 1 leader_uuid: "2ee2da566691468489827d1f7227ee7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ee2da566691468489827d1f7227ee7e" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 41029 } health_report { overall_health: HEALTHY } } }
I20260806 01:02:03.697125 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.692231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.5.108.130:41029)
0806 01:02:03.692473 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.692477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.692495 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:02:03.692677 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.692680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.692763 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.692881 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:02:03.693217 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.693885 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.695118 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.695564 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.695657 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.696058 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:02:03.696542 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:02:03.696659 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.696757 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.696819 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 01:02:03.705232 21288 raft_consensus.cc:1261] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [term 1 FOLLOWER]: Refusing update from remote peer 2ee2da566691468489827d1f7227ee7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:02:03.705240 21701 raft_consensus.cc:1261] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [term 1 FOLLOWER]: Refusing update from remote peer 2ee2da566691468489827d1f7227ee7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:02:03.706313 21809 consensus_queue.cc:1035] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1951a3f81efa467393b5e6852ea28a85" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 44613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:02:03.706692 21804 consensus_queue.cc:1035] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "75c357abc48947f9a98a9b0c868fa630" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:02:03.719713 21815 mvcc.cc:205] Tried to move back new op lower bound from 7315366394685550592 to 7315366394402308096. Current Snapshot: MvccSnapshot[applied={T|T < 7315366394685550592}]
I20260806 01:02:03.734897 21084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:02:03.735203 21084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:02:03.738886 21084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55bebab39cb14a7bb2b26676aee52cb4: Sending DeleteTablet for 3 replicas of tablet 084d5d41f8e5434fbb9be2ea9f27c25c
I20260806 01:02:03.739965 21258 tablet_service.cc:1514] Processing DeleteTablet for tablet 084d5d41f8e5434fbb9be2ea9f27c25c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:02:03 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:58258
I20260806 01:02:03.740269 21668 tablet_service.cc:1514] Processing DeleteTablet for tablet 084d5d41f8e5434fbb9be2ea9f27c25c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:02:03 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:41660
I20260806 01:02:03.740302 21459 tablet_service.cc:1514] Processing DeleteTablet for tablet 084d5d41f8e5434fbb9be2ea9f27c25c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:02:03 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:38426
I20260806 01:02:03.742609 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 21172
I20260806 01:02:03.751091 21830 tablet_replica.cc:330] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: stopping tablet replica
I20260806 01:02:03.751708 21830 raft_consensus.cc:2229] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:02:03.752633 21830 raft_consensus.cc:2258] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:02:03.755281 21829 tablet_replica.cc:330] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: stopping tablet replica
I20260806 01:02:03.755769 21829 raft_consensus.cc:2229] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:02:03.756631 21829 raft_consensus.cc:2258] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:02:03.756919 21830 ts_tablet_manager.cc:1905] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:02:03.757591 21831 tablet_replica.cc:330] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: stopping tablet replica
I20260806 01:02:03.757973 21831 raft_consensus.cc:2229] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:02:03.758437 21831 raft_consensus.cc:2258] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:02:03.760883 21831 ts_tablet_manager.cc:1905] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:02:03.762100 21829 ts_tablet_manager.cc:1905] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:02:03.768603 21831 ts_tablet_manager.cc:1918] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:02:03.768796 21831 log.cc:1196] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/084d5d41f8e5434fbb9be2ea9f27c25c
I20260806 01:02:03.769400 21831 ts_tablet_manager.cc:1939] T 084d5d41f8e5434fbb9be2ea9f27c25c P 1951a3f81efa467393b5e6852ea28a85: Deleting consensus metadata
I20260806 01:02:03.769382 21829 ts_tablet_manager.cc:1918] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:02:03.769672 21829 log.cc:1196] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/084d5d41f8e5434fbb9be2ea9f27c25c
I20260806 01:02:03.770284 21829 ts_tablet_manager.cc:1939] T 084d5d41f8e5434fbb9be2ea9f27c25c P 75c357abc48947f9a98a9b0c868fa630: Deleting consensus metadata
Aug 06 01:02:03 dist-test-slave-x4qh krb5kdc[21019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:02:03.771538 21067 catalog_manager.cc:4841] TS 1951a3f81efa467393b5e6852ea28a85 (127.5.108.131:44613): tablet 084d5d41f8e5434fbb9be2ea9f27c25c (table test-table [id=91b465ac54df48e6ac4e49553fad770e]) successfully deleted
I20260806 01:02:03.772980 21065 catalog_manager.cc:4841] TS 75c357abc48947f9a98a9b0c868fa630 (127.5.108.129:35875): tablet 084d5d41f8e5434fbb9be2ea9f27c25c (table test-table [id=91b465ac54df48e6ac4e49553fad770e]) successfully deleted
I20260806 01:02:03.773113 21830 ts_tablet_manager.cc:1918] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:02:03.773303 21830 log.cc:1196] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/084d5d41f8e5434fbb9be2ea9f27c25c
I20260806 01:02:03.774241 21830 ts_tablet_manager.cc:1939] T 084d5d41f8e5434fbb9be2ea9f27c25c P 2ee2da566691468489827d1f7227ee7e: Deleting consensus metadata
I20260806 01:02:03.776064 21067 catalog_manager.cc:4841] TS 2ee2da566691468489827d1f7227ee7e (127.5.108.130:41029): tablet 084d5d41f8e5434fbb9be2ea9f27c25c (table test-table [id=91b465ac54df48e6ac4e49553fad770e]) successfully deleted
I20260806 01:02:03.778905 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.744301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58264 (local address 127.5.108.129:35875)
0806 01:02:03.744549 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.744558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.749047 (+ 4489us) server_negotiation.cc:407] Connection header received
0806 01:02:03.749233 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.749242 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.749351 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.749478 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:03.757117 (+ 7639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.758283 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.760768 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.761419 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.761531 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.771750 (+ 10219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:03.771764 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:03.771782 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:03.771831 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:03.775923 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:03.776728 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:03.776736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:03.776738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:03.776874 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:03.777308 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:03.777311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:03.777313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:03.777674 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:03.778001 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.778328 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.778547 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 01:02:03.779672 21352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:02:03.870191 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 21172
I20260806 01:02:03.898110 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 21365
Aug 06 01:02:03 dist-test-slave-x4qh krb5kdc[21019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:02:03.916239 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:03.898946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38444 (local address 127.5.108.130:41029)
0806 01:02:03.899132 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:03.899137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:03.899159 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:02:03.899422 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:03.899430 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:03.899565 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:03.899681 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:03.901414 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.902275 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.904159 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:03.904640 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:03.904804 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:03.909839 (+ 5035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:03.909855 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:03.909871 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:03.909906 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:03.913117 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:03.913815 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:03.913868 (+ 53us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:03.913871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:03.914046 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:03.914443 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:03.914448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:03.914452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:03.914835 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:03.915190 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:03.915692 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 01:02:03.915918 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:02:03.917114 21557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:02:04.031037 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 21365
I20260806 01:02:04.057561 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 21571
Aug 06 01:02:04 dist-test-slave-x4qh krb5kdc[21019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:02:04.072170 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:04.058328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41676 (local address 127.5.108.131:44613)
0806 01:02:04.058491 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:04.058496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:04.058514 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:02:04.058669 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:04.058673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:04.058740 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:04.058826 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:04.060227 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:04.060965 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:04.062183 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:04.062549 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:04.062640 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:04.066886 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:04.066899 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:04.066915 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:04.066952 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:04.069674 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:04.070246 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:04.070253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:04.070255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:04.070404 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:04.070718 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:04.070721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:04.070723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:04.071049 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:04.071271 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:04.071719 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 01:02:04.071900 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:02:04.072803 21760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:02:04.160375 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 21571
I20260806 01:02:04.183702 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 21039
I20260806 01:02:04.185213 21127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:02:04.295487 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 21039
2026-08-06T01:02:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4620 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:02:04 dist-test-slave-x4qh krb5kdc[21839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:02:04 dist-test-slave-x4qh krb5kdc[21839](info): set up 2 sockets
Aug 06 01:02:04 dist-test-slave-x4qh krb5kdc[21839](info): commencing operation
krb5kdc: starting...
W20260806 01:02:05.614161 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:02:05 dist-test-slave-x4qh krb5kdc[21839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:02:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:02:05Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.108.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.108.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/rangeradmin_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/rangeradmin_127.5.108.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.108.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.108.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/rangerlookup_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/rangerlookup_127.5.108.148@KRBTEST.COM.keytab.
W20260806 01:02:06.106343 5554 mini_kdc.cc:262] Time spent creating service keytab for rangerlookup/127.5.108.148@KRBTEST.COM: real 0.103s user 0.000s sys 0.038s
WARNING: no policy specified for HTTP/127.5.108.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.108.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/HTTP_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/HTTP_127.5.108.148@KRBTEST.COM.keytab.
I20260806 01:02:06.201419 5554 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskq1gvHq/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/postgres -l logfile start
2026-08-06 01:02:09.366 UTC [21886] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 01:02:09.366 UTC [21886] LOG: listening on IPv4 address "127.5.108.148", port 47625
2026-08-06 01:02:09.370 UTC [21886] LOG: listening on Unix socket "/tmp/.s.PGSQL.47625"
2026-08-06 01:02:09.393 UTC [21889] LOG: database system was shut down at 2026-08-06 01:02:08 UTC
2026-08-06 01:02:09.397 UTC [21886] LOG: database system is ready to accept connections
I20260806 01:02:10.558061 5554 mini_postgres.cc:96] Postgres bound to 47625
2026-08-06 01:02:10.585 UTC [21899] FATAL: database "slave" does not exist
127.5.108.148:47625 - accepting connections
I20260806 01:02:10.586162 5554 mini_ranger.cc:161] Starting Ranger...
I20260806 01:02:10.619293 5554 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 01:02:11.059288 5554 mini_ranger.cc:87] Created ranger Postgres database
I20260806 01:02:11.059506 5554 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-admin
2026-08-06 01:02:11,463 [I] DB FLAVOR :POSTGRES
2026-08-06 01:02:11,464 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:02:11,464 [I] Checking connection..
2026-08-06 01:02:11,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 01:02:11,738 [I] Checking connection passed.
2026-08-06 01:02:11,738 [I] --------- Verifying version history table ---------
2026-08-06 01:02:11,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:02:11,982 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:02:11,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:02:12,240 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:02:12,241 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 01:02:12,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 01:02:12.502 UTC [21987] WARNING: there is no transaction in progress
2026-08-06 01:02:12,510 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 01:02:12,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:02:12,764 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 01:02:12,764 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:02:12,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 01:02:13,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 01:02:13,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x4qh.c.gcp-upstream.internal','N') ;"
2026-08-06 01:02:13,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:02:13,775 [I] Table x_portal_user does not exist in database ranger
2026-08-06 01:02:13,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:02:14,002 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 01:02:14,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:02:14,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:02:14,512 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 01:02:14,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 01:02:15.408 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.423 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.438 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.452 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.466 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.483 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.538 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.545 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.558 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.568 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.577 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.590 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.598 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.604 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.610 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:15.615 UTC [22142] WARNING: there is no transaction in progress
2026-08-06 01:02:16,193 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 01:02:16,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 01:02:16,465 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 01:02:16,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:02:16,717 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 01:02:16,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:02:16,983 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 01:02:16,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:02:17,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:02:17,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x4qh.c.gcp-upstream.internal';"
2026-08-06 01:02:17,719 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 01:02:17,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 01:02:17,950 [I] DB_PATCHES have already been applied
I20260806 01:02:17.956519 5554 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-admin:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskq1gvHq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 01:02:17.956634 5554 mini_ranger.cc:193] Using host: 127.5.108.148
I20260806 01:02:17.979960 5554 mini_ranger.cc:227] Ranger admin URL: http://127.5.108.148:34121
Aug 06, 2026 1:02:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:02:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:02:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 1:02:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 1:02:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 1:02:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 1:02:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 01:02:19.395293 5554 mini_ranger.cc:160] Time spent starting Ranger: real 8.809s user 0.000s sys 0.066s
Aug 06, 2026 1:02:38 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 1:02:38 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 1:02:38 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 1:02:38 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 1:02:39 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 01:02:39.836450 5554 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.108.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.108.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/rangerkms_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/rangerkms_127.5.108.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.108.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.108.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.108.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/HTTP_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.108.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/HTTP_127.5.108.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 01:02:40 dist-test-slave-x4qh krb5kdc[21839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978160, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 01:02:40.076246 5554 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 01:02:40.103605 5554 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 01:02:40.821540 5554 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 01:02:40.821718 5554 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-kms
I20260806 01:02:40.821875 5554 mini_ranger_kms.cc:227] Using postgres at 127.5.108.148:47625
2026-08-06 01:02:41,353 [I] DB FLAVOR :POSTGRES
2026-08-06 01:02:41,353 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:02:41,353 [I] Checking connection
2026-08-06 01:02:41,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 01:02:41,616 [I] connection success
2026-08-06 01:02:41,616 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 01:02:41,616 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 01:02:41,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 01:02:41,847 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 01:02:41,847 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:02:41,847 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 01:02:41,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:47625/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 01:02:42,100 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 01:02:42.543991 5554 mini_ranger_kms.cc:310] Created kms service
I20260806 01:02:42.717904 5554 mini_ranger_kms.cc:326] Created kudu user
I20260806 01:02:42.762692 5554 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 01:02:42 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978160, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:02:43.180423 5554 mini_ranger_kms.cc:384] Added ranger policy
I20260806 01:02:43.181738 5554 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-kms:/tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 01:02:43.181824 5554 mini_ranger_kms.cc:240] Using host: 127.5.108.148
I20260806 01:02:43.209599 5554 mini_ranger_kms.cc:276] Ranger KMS PID: 22393
I20260806 01:02:43.209790 5554 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.108.148:41613
Aug 06, 2026 1:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 1:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 1:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 1:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 01:02:45 dist-test-slave-x4qh krb5kdc[21839](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978165, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.108.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 01:02:45 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978165, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.108.148@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:02:46.006357 5554 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.930s user 0.126s sys 0.445s
I20260806 01:02:46.006768 5554 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 01:02:46.007004 5554 mini_ranger_kms.cc:401] 127.5.108.148:41613/kms/v1/keys
Aug 06 01:02:46 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978165, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.108.148@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
Aug 06 01:02:48 dist-test-slave-x4qh krb5kdc[21839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:02:48.802510 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:35357
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:35357
--ranger_config_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:41613/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:02:49.157671 22464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:02:49.157964 22464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:02:49.158082 22464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:02:49.165180 22464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:02:49.165313 22464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:02:49.165413 22464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:02:49.165518 22464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:02:49.165598 22464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:02:49.175627 22464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.108.148:41613/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:35357
--ranger_config_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:35357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:02:49.177057 22464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:02:49.178310 22464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:02:49.190794 22471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:49.190847 22472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:49.190814 22474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:02:49.191530 22464 server_base.cc:1034] running on GCE node
I20260806 01:02:49.192464 22464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:02:49.193791 22464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:02:49.194991 22464 hybrid_clock.cc:648] HybridClock initialized: now 1785978169194944 us; error 33 us; skew 500 ppm
I20260806 01:02:49.195358 22464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:02:49 dist-test-slave-x4qh krb5kdc[21839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978169, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:02:49.198582 22464 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:02:49.200002 22464 webserver.cc:460] Webserver started at http://127.5.108.190:41471/ using document root <none> and password file <none>
I20260806 01:02:49.200591 22464 fs_manager.cc:359] Metadata directory not provided
I20260806 01:02:49.200688 22464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:02:49.200956 22464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:02:49 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978169, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:02:49.251299 22464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "15b1c921574f4da384e5f884c2dd50c7"
format_stamp: "Formatted at 2026-08-06 01:02:49 on dist-test-slave-x4qh"
server_key: "1c866f41c746abb32ea8c3121ecac5d8"
server_key_iv: "8e222c12d7e318178a8e0f1742dba75a"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:49.252244 22464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "15b1c921574f4da384e5f884c2dd50c7"
format_stamp: "Formatted at 2026-08-06 01:02:49 on dist-test-slave-x4qh"
server_key: "1c866f41c746abb32ea8c3121ecac5d8"
server_key_iv: "8e222c12d7e318178a8e0f1742dba75a"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:49.257973 22464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:02:49.278440 22484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:49.280301 22464 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 01:02:49.280506 22464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "15b1c921574f4da384e5f884c2dd50c7"
format_stamp: "Formatted at 2026-08-06 01:02:49 on dist-test-slave-x4qh"
server_key: "1c866f41c746abb32ea8c3121ecac5d8"
server_key_iv: "8e222c12d7e318178a8e0f1742dba75a"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:49.280783 22464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:02:49.319698 22464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:02:49.324604 22464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:02:49.324957 22464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:02:49.344741 22464 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:35357
I20260806 01:02:49.344744 22552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:35357 every 8 connection(s)
I20260806 01:02:49.346467 22464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:02:49.349797 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 22464
I20260806 01:02:49.350003 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:02:49.350878 22553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:02:49.350871 5554 external_mini_cluster.cc:1428] Setting key 36ac456bed6c81990482e93834e0eff2
I20260806 01:02:49.366382 22553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7: Bootstrap starting.
Aug 06 01:02:49 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:02:49.370479 22553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7: Neither blocks nor log segments found. Creating new log.
I20260806 01:02:49.371634 22553 log.cc:824] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:02:49.375340 22553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7: No bootstrap required, opened a new log
I20260806 01:02:49.380645 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:49.353528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51366 (local address 127.5.108.190:35357)
0806 01:02:49.356035 (+ 2507us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:49.356051 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:49.356104 (+ 53us) server_negotiation.cc:407] Connection header received
0806 01:02:49.358635 (+ 2531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:49.358678 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:49.359236 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:49.359762 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:49.361322 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:49.363330 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:49.365237 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:49.366276 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:49.366464 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:49.371519 (+ 5055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:49.371531 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:49.371550 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:49.371597 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:49.375635 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:49.376476 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:49.376484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:49.376493 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:49.376707 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:49.377214 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:49.377219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:49.377222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:49.377970 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:49.378304 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:49.378802 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 01:02:49.379211 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2269,"thread_start_us":2021,"threads_started":1}
I20260806 01:02:49.384950 22553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b1c921574f4da384e5f884c2dd50c7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35357 } }
I20260806 01:02:49.385443 22553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:02:49.385571 22553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b1c921574f4da384e5f884c2dd50c7, State: Initialized, Role: FOLLOWER
I20260806 01:02:49.386157 22553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b1c921574f4da384e5f884c2dd50c7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35357 } }
I20260806 01:02:49.386438 22553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:02:49.386582 22553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:02:49.386780 22553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:02:49.389261 22553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b1c921574f4da384e5f884c2dd50c7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35357 } }
I20260806 01:02:49.389724 22553 leader_election.cc:302] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15b1c921574f4da384e5f884c2dd50c7; no voters:
I20260806 01:02:49.390425 22553 leader_election.cc:288] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:02:49.390548 22560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:02:49.391860 22560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [term 1 LEADER]: Becoming Leader. State: Replica: 15b1c921574f4da384e5f884c2dd50c7, State: Running, Role: LEADER
I20260806 01:02:49.392485 22560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b1c921574f4da384e5f884c2dd50c7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35357 } }
I20260806 01:02:49.392910 22553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:02:49.401688 22561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15b1c921574f4da384e5f884c2dd50c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b1c921574f4da384e5f884c2dd50c7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35357 } } }
I20260806 01:02:49.402067 22561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:02:49.400884 22562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15b1c921574f4da384e5f884c2dd50c7. Latest consensus state: current_term: 1 leader_uuid: "15b1c921574f4da384e5f884c2dd50c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b1c921574f4da384e5f884c2dd50c7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35357 } } }
I20260806 01:02:49.402684 22562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:02:49.407024 22577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:02:49.407653 22553 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 01:02:49.430233 22577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:02:49.451306 22577 catalog_manager.cc:1347] Generated new cluster ID: 89cf5eb8a5384ce8bfa707d8c2ebb955
I20260806 01:02:49.451462 22577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:02:49.464751 22577 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:02:49.467468 22577 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:02:49.474967 22577 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7: Generated new TSK 0
I20260806 01:02:49.476027 22577 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:02:50.259742 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:41613/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35357
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:02:50 dist-test-slave-x4qh krb5kdc[21839](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 01:02:50.718900 22617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:02:50.719414 22617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:02:50.719631 22617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:02:50.731452 22617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:02:50.731609 22617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:02:50.731819 22617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:02:50.747642 22617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.108.148:41613/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:02:50.749361 22617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:02:50.751284 22617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:02:50.769579 22630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:50.772120 22629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:50.776491 22632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:02:50.777464 22617 server_base.cc:1034] running on GCE node
I20260806 01:02:50.778532 22617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:02:50.780665 22617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:02:50.781988 22617 hybrid_clock.cc:648] HybridClock initialized: now 1785978170781882 us; error 93 us; skew 500 ppm
I20260806 01:02:50.782433 22617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:02:50 dist-test-slave-x4qh krb5kdc[21839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:02:50.787149 22617 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:02:50.792116 22617 webserver.cc:460] Webserver started at http://127.5.108.129:41879/ using document root <none> and password file <none>
I20260806 01:02:50.793300 22617 fs_manager.cc:359] Metadata directory not provided
I20260806 01:02:50.793510 22617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:02:50.793982 22617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:02:50 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:02:50.821202 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "ad9c2989db564dc883f097b98d988fe9"
format_stamp: "Formatted at 2026-08-06 01:02:50 on dist-test-slave-x4qh"
server_key: "6ead2f1260ef362548131ee7b2e7c783"
server_key_iv: "86bdc08ddece69b6b0c3c5f9cc834096"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:50.822144 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "ad9c2989db564dc883f097b98d988fe9"
format_stamp: "Formatted at 2026-08-06 01:02:50 on dist-test-slave-x4qh"
server_key: "6ead2f1260ef362548131ee7b2e7c783"
server_key_iv: "86bdc08ddece69b6b0c3c5f9cc834096"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:50.828121 22617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 01:02:50.846400 22644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:50.848106 22617 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 01:02:50.848304 22617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "ad9c2989db564dc883f097b98d988fe9"
format_stamp: "Formatted at 2026-08-06 01:02:50 on dist-test-slave-x4qh"
server_key: "6ead2f1260ef362548131ee7b2e7c783"
server_key_iv: "86bdc08ddece69b6b0c3c5f9cc834096"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:50.848543 22617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:02:50.932945 22617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:02:50.938800 22617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:02:50.939260 22617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:02:50.951889 22617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:02:50.961614 22617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:02:50.961841 22617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:50.962082 22617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:02:50.962234 22617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:02:51 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:02:51.308420 22617 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:35571
I20260806 01:02:51.308435 22798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:35571 every 8 connection(s)
I20260806 01:02:51.310088 22617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:02:51.313371 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 22617
I20260806 01:02:51.313593 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:02:51.314445 5554 external_mini_cluster.cc:1428] Setting key 448705384ac51c0f623934cd98cdeda9
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Aug 06 01:02:51 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:02:51.370266 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:51.313209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:56659 (local address 127.5.108.190:35357)
0806 01:02:51.330456 (+ 17247us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:51.330467 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:51.332751 (+ 2284us) server_negotiation.cc:407] Connection header received
0806 01:02:51.335741 (+ 2990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:51.335747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:51.335853 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:51.335932 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:51.338167 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:51.339316 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:51.341796 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:51.342408 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:51.342538 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:51.361953 (+ 19415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:51.361962 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:51.361967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:51.362016 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:51.365936 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:51.367092 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:51.367098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:51.367102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:51.367264 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:51.367899 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:51.367904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:51.367907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:51.368302 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:51.368431 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:51.369603 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0806 01:02:51.369867 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17048,"thread_start_us":17250,"threads_started":1}
I20260806 01:02:51.372969 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:51.313898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:35357 (local address 127.5.108.129:56659)
0806 01:02:51.332566 (+ 18668us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:51.332618 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:51.335363 (+ 2745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:51.336491 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:51.336502 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:51.337131 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:02:51.337954 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:51.337983 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:51.339564 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:51.339569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:51.341605 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:51.341621 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:51.342928 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:51.342935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:51.343205 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:51.354181 (+ 10976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:02:51.354226 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:02:51.361598 (+ 7372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:02:51.366173 (+ 4575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:51.366184 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:51.366202 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:51.366930 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:51.367451 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:51.367456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:51.367460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:51.367751 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:51.368605 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:02:51.368614 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:02:51.369119 (+ 505us) client_negotiation.cc:769] Sending connection context
0806 01:02:51.370853 (+ 1734us) client_negotiation.cc:240] Negotiation successful
0806 01:02:51.371482 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18355,"thread_start_us":18098,"threads_started":1}
I20260806 01:02:51.375216 22799 heartbeater.cc:347] Connected to a master server at 127.5.108.190:35357
I20260806 01:02:51.375551 22799 heartbeater.cc:464] Registering TS with master...
I20260806 01:02:51.376386 22799 heartbeater.cc:511] Master 127.5.108.190:35357 requested a full tablet report, sending...
I20260806 01:02:51.379180 22509 ts_manager.cc:194] Registered new tserver with Master: ad9c2989db564dc883f097b98d988fe9 (127.5.108.129:35571)
I20260806 01:02:51.381228 22509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:56659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:02:51.396965 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:41613/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35357
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:02:51 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
W20260806 01:02:51.934520 22805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:02:51.934981 22805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:02:51.935199 22805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:02:51.948477 22805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:02:51.948661 22805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:02:51.948935 22805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:02:51.968401 22805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.108.148:41613/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:02:51.970235 22805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:02:51.972205 22805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:02:52.001273 22825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:52.001456 22829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:52.001678 22827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:02:52.004973 22805 server_base.cc:1034] running on GCE node
I20260806 01:02:52.005762 22805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:02:52.007128 22805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:02:52.008657 22805 hybrid_clock.cc:648] HybridClock initialized: now 1785978172008456 us; error 116 us; skew 500 ppm
I20260806 01:02:52.009119 22805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:02:52 dist-test-slave-x4qh krb5kdc[21839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:02:52.016855 22805 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:02:52.018956 22805 webserver.cc:460] Webserver started at http://127.5.108.130:44067/ using document root <none> and password file <none>
I20260806 01:02:52.019905 22805 fs_manager.cc:359] Metadata directory not provided
I20260806 01:02:52.020140 22805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:02:52.020601 22805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:02:52 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:02:52.040791 22805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "f161d00e68184f59946b36e97ef7a9dd"
format_stamp: "Formatted at 2026-08-06 01:02:52 on dist-test-slave-x4qh"
server_key: "9c0d0ffc3c1fd70f69600de87c6f9859"
server_key_iv: "6e4233a285c3dd0a65317f3f4441c7fc"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:52.042217 22805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "f161d00e68184f59946b36e97ef7a9dd"
format_stamp: "Formatted at 2026-08-06 01:02:52 on dist-test-slave-x4qh"
server_key: "9c0d0ffc3c1fd70f69600de87c6f9859"
server_key_iv: "6e4233a285c3dd0a65317f3f4441c7fc"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:52.050918 22805 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.008s
I20260806 01:02:52.069235 22839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:52.071123 22805 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:02:52.071439 22805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "f161d00e68184f59946b36e97ef7a9dd"
format_stamp: "Formatted at 2026-08-06 01:02:52 on dist-test-slave-x4qh"
server_key: "9c0d0ffc3c1fd70f69600de87c6f9859"
server_key_iv: "6e4233a285c3dd0a65317f3f4441c7fc"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:52.071792 22805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:02:52.113371 22805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:02:52.118793 22805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:02:52.119246 22805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:02:52.120899 22805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:02:52.123566 22805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:02:52.123724 22805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:02:52.123916 22805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:02:52.124022 22805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:52.229797 22805 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:42993
I20260806 01:02:52.229967 22986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:42993 every 8 connection(s)
I20260806 01:02:52.231531 22805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:02:52.239351 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 22805
I20260806 01:02:52.239560 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:02:52.240127 5554 external_mini_cluster.cc:1428] Setting key b62725d61635fd25434a27c25645b273
Aug 06 01:02:52 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:02:52.259886 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:52.235936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:37259 (local address 127.5.108.190:35357)
0806 01:02:52.236907 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:52.236912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:52.238330 (+ 1418us) server_negotiation.cc:407] Connection header received
0806 01:02:52.242239 (+ 3909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:52.242244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:52.242358 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:52.242468 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:52.244246 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:52.245273 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:52.247445 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:52.247953 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:52.248085 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:52.252418 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:52.252423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:52.252426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:52.252457 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:52.256192 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:52.256993 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:52.257002 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:52.257007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:52.257178 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:52.257707 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:52.257715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:52.257721 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:52.258153 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:52.258333 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:52.259107 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 01:02:52.259330 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":690,"threads_started":1}
I20260806 01:02:52.261031 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:52.236741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:35357 (local address 127.5.108.130:37259)
0806 01:02:52.238163 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:52.238209 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:52.241301 (+ 3092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:52.242778 (+ 1477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:52.242786 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:52.243312 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:02:52.244079 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:52.244096 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:52.245420 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:52.245424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:52.247278 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:52.247291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:52.248322 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:52.248334 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:52.248640 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:52.249495 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:02:52.249517 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:02:52.252195 (+ 2678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:02:52.256319 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:52.256328 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:52.256343 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:52.256863 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:52.257315 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:52.257320 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:52.257324 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:52.257584 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:52.258313 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:02:52.258320 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:02:52.258668 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 01:02:52.258962 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 01:02:52.259604 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"thread_start_us":735,"threads_started":1}
I20260806 01:02:52.262595 22987 heartbeater.cc:347] Connected to a master server at 127.5.108.190:35357
I20260806 01:02:52.262962 22987 heartbeater.cc:464] Registering TS with master...
I20260806 01:02:52.263947 22987 heartbeater.cc:511] Master 127.5.108.190:35357 requested a full tablet report, sending...
I20260806 01:02:52.265601 22509 ts_manager.cc:194] Registered new tserver with Master: f161d00e68184f59946b36e97ef7a9dd (127.5.108.130:42993)
I20260806 01:02:52.266489 22509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:37259
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:02:52.320905 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:41613/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35357
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:02:52.385011 22799 heartbeater.cc:503] Master 127.5.108.190:35357 was elected leader, sending a full tablet report...
W20260806 01:02:52.839597 22994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:02:52.839960 22994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:02:52.840127 22994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:02:52.848095 22994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:02:52.848263 22994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:02:52.848465 22994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:02:52.860261 22994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.108.148:41613/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:35357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:02:52.861625 22994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:02:52.862993 22994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:02:52.878177 23001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:52.878600 23000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:02:52.879230 23004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:02:52.879420 22994 server_base.cc:1034] running on GCE node
I20260806 01:02:52.880015 22994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:02:52.880905 22994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:02:52.882126 22994 hybrid_clock.cc:648] HybridClock initialized: now 1785978172882078 us; error 56 us; skew 500 ppm
I20260806 01:02:52.882474 22994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:02:52 dist-test-slave-x4qh krb5kdc[21839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:02:52.886109 22994 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:02:52.888041 22994 webserver.cc:460] Webserver started at http://127.5.108.131:44469/ using document root <none> and password file <none>
I20260806 01:02:52.888770 22994 fs_manager.cc:359] Metadata directory not provided
I20260806 01:02:52.888947 22994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:02:52.889302 22994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:02:52 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:02:52.904241 22994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "0bd199fcf31445ceaa94f51280029e20"
format_stamp: "Formatted at 2026-08-06 01:02:52 on dist-test-slave-x4qh"
server_key: "b27306021445d714858b6977b34ea16f"
server_key_iv: "0a82099cfca7074bfc9f8785bc28972a"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:52.905222 22994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "0bd199fcf31445ceaa94f51280029e20"
format_stamp: "Formatted at 2026-08-06 01:02:52 on dist-test-slave-x4qh"
server_key: "b27306021445d714858b6977b34ea16f"
server_key_iv: "0a82099cfca7074bfc9f8785bc28972a"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:52.911291 22994 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 01:02:52.929863 23014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:52.931452 22994 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260806 01:02:52.931663 22994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "0bd199fcf31445ceaa94f51280029e20"
format_stamp: "Formatted at 2026-08-06 01:02:52 on dist-test-slave-x4qh"
server_key: "b27306021445d714858b6977b34ea16f"
server_key_iv: "0a82099cfca7074bfc9f8785bc28972a"
server_key_version: "kuduclusterkey@0"
I20260806 01:02:52.931896 22994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:02:52.957139 22994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:02:52.961550 22994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:02:52.961905 22994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:02:52.963315 22994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:02:52.965109 22994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:02:52.965181 22994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:52.965309 22994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:02:52.965427 22994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:02:53.027283 22994 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:39571
I20260806 01:02:53.027307 23168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:39571 every 8 connection(s)
I20260806 01:02:53.029299 22994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:02:53.039667 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 22994
I20260806 01:02:53.039882 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:02:53.040448 5554 external_mini_cluster.cc:1428] Setting key 98592c283e6ffd3eafa1435d99648b45
Aug 06 01:02:53 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978172, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:02:53.055735 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.033281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:51883 (local address 127.5.108.190:35357)
0806 01:02:53.033809 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.033816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.035433 (+ 1617us) server_negotiation.cc:407] Connection header received
0806 01:02:53.037766 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.037772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.037869 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.038000 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:53.039834 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.041247 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.043445 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.044361 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.044500 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.048786 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:53.048792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:53.048795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:53.048828 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:53.051988 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.052811 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.052816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.052820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.052941 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.053508 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.053514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.053517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.053879 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:53.054076 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.055026 (+ 950us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.055320 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":240,"threads_started":1}
I20260806 01:02:53.057106 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.033986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:35357 (local address 127.5.108.131:51883)
0806 01:02:53.035248 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.035296 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:53.037490 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:53.038211 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:53.038219 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:53.038717 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:02:53.039571 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.039597 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.041471 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.041476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.043269 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.043281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.044712 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.044723 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.045109 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.045920 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:02:53.045947 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:02:53.048550 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:02:53.052225 (+ 3675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:53.052235 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:53.052251 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:53.052677 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:53.053051 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:02:53.053055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:02:53.053059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:02:53.053373 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:02:53.054035 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:02:53.054042 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:02:53.054501 (+ 459us) client_negotiation.cc:769] Sending connection context
0806 01:02:53.054851 (+ 350us) client_negotiation.cc:240] Negotiation successful
0806 01:02:53.055464 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"thread_start_us":764,"threads_started":1}
I20260806 01:02:53.058588 23169 heartbeater.cc:347] Connected to a master server at 127.5.108.190:35357
I20260806 01:02:53.058897 23169 heartbeater.cc:464] Registering TS with master...
I20260806 01:02:53.059662 23169 heartbeater.cc:511] Master 127.5.108.190:35357 requested a full tablet report, sending...
I20260806 01:02:53.061124 22509 ts_manager.cc:194] Registered new tserver with Master: 0bd199fcf31445ceaa94f51280029e20 (127.5.108.131:39571)
I20260806 01:02:53.061992 22509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:51883
I20260806 01:02:53.069310 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:02:53.092564 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.075604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.5.108.190:35357)
0806 01:02:53.075814 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.075832 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.076519 (+ 687us) server_negotiation.cc:407] Connection header received
0806 01:02:53.076726 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.076730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.076804 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.076931 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:53.079198 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.080025 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.082261 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.082736 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.082900 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.086310 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:53.086315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:53.086318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:53.086351 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:53.089467 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.090354 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.090358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.090360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.090488 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.090991 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.090994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.090996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.091316 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:53.091606 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.092039 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.092260 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:02:53.097038 22509 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:02:53.105314 22509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:02:53.136797 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.119978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.0.0.1:44456)
0806 01:02:53.120927 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.120967 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:53.121204 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:53.125069 (+ 3865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:53.125076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:53.125107 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:53.125777 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.125792 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.129403 (+ 3611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.129410 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.132923 (+ 3513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.132960 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.135579 (+ 2619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.135586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.135781 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.135865 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 01:02:53.135988 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 01:02:53.136302 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":929,"threads_started":1}
I20260806 01:02:53.138696 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.121436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.5.108.130:42993)
0806 01:02:53.122198 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.122208 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.124693 (+ 2485us) server_negotiation.cc:407] Connection header received
0806 01:02:53.125030 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.125038 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.125289 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.125497 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:53.126866 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.128655 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.135048 (+ 6393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.136937 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.137062 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.137278 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.138077 (+ 799us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.138289 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":1166,"threads_started":1}
I20260806 01:02:53.139070 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.120804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44456 (local address 127.5.108.129:35571)
0806 01:02:53.124198 (+ 3394us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.124219 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.124269 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:02:53.124408 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.124416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.124747 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.126004 (+ 1257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:53.126451 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.128566 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.133221 (+ 4655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.135216 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.137756 (+ 2540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.137997 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.138070 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.138351 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3080,"thread_start_us":2874,"threads_started":1}
I20260806 01:02:53.139632 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.120618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:42993 (local address 127.0.0.1:56666)
0806 01:02:53.124556 (+ 3938us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.124575 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:53.124805 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:53.125816 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:53.125827 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:53.125853 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:53.126621 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.126638 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.128977 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.128984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.134826 (+ 5842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.134844 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.137592 (+ 2748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.137601 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.137829 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.137870 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 01:02:53.138919 (+ 1049us) client_negotiation.cc:240] Negotiation successful
0806 01:02:53.139076 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3807,"mutex_wait_us":3,"thread_start_us":3652,"threads_started":1}
I20260806 01:02:53.143467 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.121713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:39571 (local address 127.0.0.1:51504)
0806 01:02:53.129402 (+ 7689us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.129425 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:53.129660 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:53.131306 (+ 1646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:53.131310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:53.131335 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:53.132549 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.132563 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.136067 (+ 3504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.136072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.139987 (+ 3915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.140001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.142278 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.142287 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.142506 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.142552 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 01:02:53.142683 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 01:02:53.142834 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7513,"mutex_wait_us":30,"thread_start_us":7278,"threads_started":1}
I20260806 01:02:53.143746 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.122315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51504 (local address 127.5.108.131:39571)
0806 01:02:53.123298 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.123310 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.129516 (+ 6206us) server_negotiation.cc:407] Connection header received
0806 01:02:53.130447 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.130456 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.130774 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.130949 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:53.132829 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.135448 (+ 2619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.140168 (+ 4720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.141921 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.142059 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.142276 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.142791 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.143393 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":565,"threads_started":1}
I20260806 01:02:53.144234 22902 tablet_service.cc:1467] Processing CreateTablet for tablet 5543097fa9d6430da124aebf27f9cb02 (DEFAULT_TABLE table=test-table [id=eeb14b45723f4e8db63a5d4a60e4c6a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:02:53.144256 22711 tablet_service.cc:1467] Processing CreateTablet for tablet 5543097fa9d6430da124aebf27f9cb02 (DEFAULT_TABLE table=test-table [id=eeb14b45723f4e8db63a5d4a60e4c6a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:02:53.145961 22711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5543097fa9d6430da124aebf27f9cb02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:02:53.145998 22902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5543097fa9d6430da124aebf27f9cb02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:02:53.149402 23078 tablet_service.cc:1467] Processing CreateTablet for tablet 5543097fa9d6430da124aebf27f9cb02 (DEFAULT_TABLE table=test-table [id=eeb14b45723f4e8db63a5d4a60e4c6a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:02:53.151934 23078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5543097fa9d6430da124aebf27f9cb02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:02:53.167863 23196 tablet_bootstrap.cc:492] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: Bootstrap starting.
I20260806 01:02:53.177069 23195 tablet_bootstrap.cc:492] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: Bootstrap starting.
I20260806 01:02:53.181129 23197 tablet_bootstrap.cc:492] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: Bootstrap starting.
I20260806 01:02:53.182772 23195 tablet_bootstrap.cc:654] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: Neither blocks nor log segments found. Creating new log.
I20260806 01:02:53.184799 23196 tablet_bootstrap.cc:654] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: Neither blocks nor log segments found. Creating new log.
I20260806 01:02:53.185040 23195 log.cc:824] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: Log is configured to *not* fsync() on all Append() calls
I20260806 01:02:53.186748 23196 log.cc:824] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: Log is configured to *not* fsync() on all Append() calls
I20260806 01:02:53.187203 23197 tablet_bootstrap.cc:654] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: Neither blocks nor log segments found. Creating new log.
I20260806 01:02:53.190593 23197 log.cc:824] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: Log is configured to *not* fsync() on all Append() calls
I20260806 01:02:53.194048 23196 tablet_bootstrap.cc:492] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: No bootstrap required, opened a new log
I20260806 01:02:53.194110 23195 tablet_bootstrap.cc:492] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: No bootstrap required, opened a new log
I20260806 01:02:53.194442 23196 ts_tablet_manager.cc:1397] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.000s
I20260806 01:02:53.194458 23195 ts_tablet_manager.cc:1397] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.011s
I20260806 01:02:53.195362 23197 tablet_bootstrap.cc:492] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: No bootstrap required, opened a new log
I20260806 01:02:53.195698 23197 ts_tablet_manager.cc:1397] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.012s
I20260806 01:02:53.204753 23196 raft_consensus.cc:348] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } }
I20260806 01:02:53.205185 23196 raft_consensus.cc:374] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:02:53.205354 23196 raft_consensus.cc:729] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9c2989db564dc883f097b98d988fe9, State: Initialized, Role: FOLLOWER
I20260806 01:02:53.205729 23195 raft_consensus.cc:348] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } }
I20260806 01:02:53.206199 23195 raft_consensus.cc:374] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:02:53.206030 23196 consensus_queue.cc:260] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } }
I20260806 01:02:53.206298 23195 raft_consensus.cc:729] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f161d00e68184f59946b36e97ef7a9dd, State: Initialized, Role: FOLLOWER
I20260806 01:02:53.206933 23195 consensus_queue.cc:260] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } }
I20260806 01:02:53.208467 22987 heartbeater.cc:503] Master 127.5.108.190:35357 was elected leader, sending a full tablet report...
I20260806 01:02:53.209122 23195 ts_tablet_manager.cc:1428] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260806 01:02:53.209522 23197 raft_consensus.cc:348] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } }
I20260806 01:02:53.210009 23197 raft_consensus.cc:374] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:02:53.210175 23197 raft_consensus.cc:729] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd199fcf31445ceaa94f51280029e20, State: Initialized, Role: FOLLOWER
I20260806 01:02:53.210829 23197 consensus_queue.cc:260] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } }
I20260806 01:02:53.212365 23196 ts_tablet_manager.cc:1428] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: Time spent starting tablet: real 0.018s user 0.010s sys 0.005s
I20260806 01:02:53.214082 23169 heartbeater.cc:503] Master 127.5.108.190:35357 was elected leader, sending a full tablet report...
I20260806 01:02:53.214191 23197 ts_tablet_manager.cc:1428] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
W20260806 01:02:53.235692 22989 tablet.cc:2369] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:02:53.281848 23171 tablet.cc:2369] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:02:53.322434 23204 raft_consensus.cc:482] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:02:53.322806 23204 raft_consensus.cc:504] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } }
I20260806 01:02:53.324924 23204 leader_election.cc:288] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad9c2989db564dc883f097b98d988fe9 (127.5.108.129:35571), 0bd199fcf31445ceaa94f51280029e20 (127.5.108.131:39571)
W20260806 01:02:53.333254 22800 tablet.cc:2369] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:02:53.335126 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.325242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:53501 (local address 127.5.108.129:35571)
0806 01:02:53.325554 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.325565 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.326190 (+ 625us) server_negotiation.cc:407] Connection header received
0806 01:02:53.326419 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.326426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.326560 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.326665 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:53.327642 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.328923 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.331896 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.333599 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.333807 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.333902 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.334318 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.334446 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:02:53.335161 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.326597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:54787 (local address 127.5.108.131:39571)
0806 01:02:53.326886 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.326891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.327028 (+ 137us) server_negotiation.cc:407] Connection header received
0806 01:02:53.327242 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.327245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.327309 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.327457 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:02:53.328196 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.329266 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.331925 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.333630 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.333848 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.333925 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.334533 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.334683 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:02:53.335319 23207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.325153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.5.108.130:53501)
0806 01:02:53.326070 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.326091 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:53.326273 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:53.326920 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:53.326925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:53.326955 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:53.327466 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.327478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.329060 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.329063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.331693 (+ 2630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.331710 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.333906 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.333914 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.334120 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.334170 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 01:02:53.334295 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 01:02:53.334431 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":31,"thread_start_us":555,"threads_started":1}
I20260806 01:02:53.335585 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.326065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:39571 (local address 127.5.108.130:54787)
0806 01:02:53.326932 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.326949 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:02:53.327111 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:02:53.327479 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:02:53.327484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:02:53.327510 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:02:53.328020 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.328029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.329404 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.329409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.331771 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:02:53.331784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.334119 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.334125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:02:53.334364 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.334391 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 01:02:53.334504 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 01:02:53.334656 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":59,"thread_start_us":529,"threads_started":1}
I20260806 01:02:53.336858 23103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5543097fa9d6430da124aebf27f9cb02" candidate_uuid: "f161d00e68184f59946b36e97ef7a9dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd199fcf31445ceaa94f51280029e20" is_pre_election: true
I20260806 01:02:53.336901 22736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5543097fa9d6430da124aebf27f9cb02" candidate_uuid: "f161d00e68184f59946b36e97ef7a9dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9c2989db564dc883f097b98d988fe9" is_pre_election: true
I20260806 01:02:53.337694 23103 raft_consensus.cc:2454] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f161d00e68184f59946b36e97ef7a9dd in term 0.
I20260806 01:02:53.337694 22736 raft_consensus.cc:2454] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f161d00e68184f59946b36e97ef7a9dd in term 0.
I20260806 01:02:53.338433 22840 leader_election.cc:302] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd199fcf31445ceaa94f51280029e20, f161d00e68184f59946b36e97ef7a9dd; no voters:
I20260806 01:02:53.338889 23204 raft_consensus.cc:2781] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:02:53.339154 23204 raft_consensus.cc:482] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:02:53.339318 23204 raft_consensus.cc:3037] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:02:53.342415 23204 raft_consensus.cc:504] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } }
I20260806 01:02:53.343312 23204 leader_election.cc:288] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [CANDIDATE]: Term 1 election: Requested vote from peers ad9c2989db564dc883f097b98d988fe9 (127.5.108.129:35571), 0bd199fcf31445ceaa94f51280029e20 (127.5.108.131:39571)
I20260806 01:02:53.343860 22736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5543097fa9d6430da124aebf27f9cb02" candidate_uuid: "f161d00e68184f59946b36e97ef7a9dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9c2989db564dc883f097b98d988fe9"
I20260806 01:02:53.343871 23103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5543097fa9d6430da124aebf27f9cb02" candidate_uuid: "f161d00e68184f59946b36e97ef7a9dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd199fcf31445ceaa94f51280029e20"
I20260806 01:02:53.344131 22736 raft_consensus.cc:3037] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:02:53.344152 23103 raft_consensus.cc:3037] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:02:53.348196 23103 raft_consensus.cc:2454] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f161d00e68184f59946b36e97ef7a9dd in term 1.
I20260806 01:02:53.348227 22736 raft_consensus.cc:2454] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f161d00e68184f59946b36e97ef7a9dd in term 1.
I20260806 01:02:53.348716 22840 leader_election.cc:302] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd199fcf31445ceaa94f51280029e20, f161d00e68184f59946b36e97ef7a9dd; no voters:
I20260806 01:02:53.349141 23204 raft_consensus.cc:2781] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:02:53.349860 23204 raft_consensus.cc:686] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 1 LEADER]: Becoming Leader. State: Replica: f161d00e68184f59946b36e97ef7a9dd, State: Running, Role: LEADER
I20260806 01:02:53.350447 23204 consensus_queue.cc:237] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } }
I20260806 01:02:53.355693 22509 catalog_manager.cc:5495] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd reported cstate change: term changed from 0 to 1, leader changed from <none> to f161d00e68184f59946b36e97ef7a9dd (127.5.108.130). New cstate: current_term: 1 leader_uuid: "f161d00e68184f59946b36e97ef7a9dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f161d00e68184f59946b36e97ef7a9dd" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 42993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 } health_report { overall_health: UNKNOWN } } }
I20260806 01:02:53.403357 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.396593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.5.108.130:42993)
0806 01:02:53.396892 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.396901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.396965 (+ 64us) server_negotiation.cc:407] Connection header received
0806 01:02:53.397246 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.397250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.397346 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.397514 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:02:53.397967 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.398888 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.400780 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.401290 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.401393 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.402073 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:02:53.402594 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:02:53.402735 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.402937 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.403049 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 01:02:53.412864 23103 raft_consensus.cc:1261] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [term 1 FOLLOWER]: Refusing update from remote peer f161d00e68184f59946b36e97ef7a9dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:02:53.412881 22736 raft_consensus.cc:1261] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [term 1 FOLLOWER]: Refusing update from remote peer f161d00e68184f59946b36e97ef7a9dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:02:53.413911 23212 consensus_queue.cc:1035] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd199fcf31445ceaa94f51280029e20" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 39571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:02:53.414456 23204 consensus_queue.cc:1035] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9c2989db564dc883f097b98d988fe9" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 35571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:02:53.443428 22507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:02:53.443861 22507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:02:53.448369 22507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15b1c921574f4da384e5f884c2dd50c7: Sending DeleteTablet for 3 replicas of tablet 5543097fa9d6430da124aebf27f9cb02
I20260806 01:02:53.449648 22902 tablet_service.cc:1514] Processing DeleteTablet for tablet 5543097fa9d6430da124aebf27f9cb02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:02:53 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:56666
I20260806 01:02:53.449658 22711 tablet_service.cc:1514] Processing DeleteTablet for tablet 5543097fa9d6430da124aebf27f9cb02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:02:53 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:44456
I20260806 01:02:53.449934 23078 tablet_service.cc:1514] Processing DeleteTablet for tablet 5543097fa9d6430da124aebf27f9cb02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:02:53 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:51504
I20260806 01:02:53.454341 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 22617
I20260806 01:02:53.459807 23231 tablet_replica.cc:330] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: stopping tablet replica
I20260806 01:02:53.460433 23231 raft_consensus.cc:2229] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:02:53.461061 23231 raft_consensus.cc:2258] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:02:53.464251 23231 ts_tablet_manager.cc:1905] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:02:53.464926 23230 tablet_replica.cc:330] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: stopping tablet replica
I20260806 01:02:53.471812 23230 raft_consensus.cc:2229] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:02:53.472764 23230 raft_consensus.cc:2258] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:02:53.475063 23230 ts_tablet_manager.cc:1905] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:02:53 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:02:53.476852 23229 tablet_replica.cc:330] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: stopping tablet replica
I20260806 01:02:53.477465 23229 raft_consensus.cc:2229] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:02:53.478112 23229 raft_consensus.cc:2258] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:02:53.478222 23231 ts_tablet_manager.cc:1918] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:02:53.478497 23231 log.cc:1196] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/5543097fa9d6430da124aebf27f9cb02
I20260806 01:02:53.479169 23231 ts_tablet_manager.cc:1939] T 5543097fa9d6430da124aebf27f9cb02 P 0bd199fcf31445ceaa94f51280029e20: Deleting consensus metadata
I20260806 01:02:53.480636 23229 ts_tablet_manager.cc:1905] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:02:53.481496 22485 catalog_manager.cc:4841] TS 0bd199fcf31445ceaa94f51280029e20 (127.5.108.131:39571): tablet 5543097fa9d6430da124aebf27f9cb02 (table test-table [id=eeb14b45723f4e8db63a5d4a60e4c6a4]) successfully deleted
I20260806 01:02:53.483964 23230 ts_tablet_manager.cc:1918] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:02:53.484259 23230 log.cc:1196] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/5543097fa9d6430da124aebf27f9cb02
I20260806 01:02:53.485105 23230 ts_tablet_manager.cc:1939] T 5543097fa9d6430da124aebf27f9cb02 P f161d00e68184f59946b36e97ef7a9dd: Deleting consensus metadata
I20260806 01:02:53.487061 22490 catalog_manager.cc:4841] TS f161d00e68184f59946b36e97ef7a9dd (127.5.108.130:42993): tablet 5543097fa9d6430da124aebf27f9cb02 (table test-table [id=eeb14b45723f4e8db63a5d4a60e4c6a4]) successfully deleted
I20260806 01:02:53.489049 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.455776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.5.108.129:35571)
0806 01:02:53.455997 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.456006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.461645 (+ 5639us) server_negotiation.cc:407] Connection header received
0806 01:02:53.461857 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.461865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.461967 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.462087 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:53.464755 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.466311 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.468964 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.469930 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.470052 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.478353 (+ 8301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:53.478372 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:53.478389 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:53.478471 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:53.482973 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.484015 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.484023 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.484026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.484256 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.484913 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.484918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.484922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.486577 (+ 1655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:53.487035 (+ 458us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.487805 (+ 770us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.488177 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:02:53.489682 22792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:02:53.589978 23229 ts_tablet_manager.cc:1918] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:02:53.590255 23229 log.cc:1196] T 5543097fa9d6430da124aebf27f9cb02 P ad9c2989db564dc883f097b98d988fe9: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/5543097fa9d6430da124aebf27f9cb02
I20260806 01:02:53.590764 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 22617
W20260806 01:02:53.615296 22485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.108.129:35571 (error 108)
I20260806 01:02:53.616240 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 22805
I20260806 01:02:53.617673 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.616973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.0.0.1:44478)
0806 01:02:53.617250 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.617400 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:35571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260806 01:02:53.618070 22485 catalog_manager.cc:4584] TS ad9c2989db564dc883f097b98d988fe9 (127.5.108.129:35571): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5543097fa9d6430da124aebf27f9cb02: Network error: Client connection negotiation failed: client connection to 127.5.108.129:35571: connect: Connection refused (error 111)
Aug 06 01:02:53 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:02:53.632937 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.617169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56690 (local address 127.5.108.130:42993)
0806 01:02:53.617355 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.617360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.617380 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:02:53.617583 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.617587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.617659 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.617781 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:53.619262 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.620167 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.621423 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.621846 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.621985 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.626510 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:53.626522 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:53.626536 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:53.626579 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:53.629831 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.630568 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.630575 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.630578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.630742 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.631153 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.631161 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.631165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.631624 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:53.631992 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.632314 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.632565 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:02:53.633670 22980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:02:53.671726 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.671176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.0.0.1:44486)
0806 01:02:53.671417 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.671563 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:35571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 01:02:53.673035 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.672612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.0.0.1:44498)
0806 01:02:53.672789 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.672898 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:35571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 01:02:53.733721 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.733139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.0.0.1:44514)
0806 01:02:53.733379 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.733494 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:35571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 01:02:53.734908 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.734510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.0.0.1:44530)
0806 01:02:53.734685 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.734772 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:35571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 01:02:53.748646 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 22805
I20260806 01:02:53.772089 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 22994
Aug 06 01:02:53 dist-test-slave-x4qh krb5kdc[21839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:02:53.787397 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.772838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51518 (local address 127.5.108.131:39571)
0806 01:02:53.773051 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 01:02:53.773056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:02:53.773074 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:02:53.773201 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:02:53.773204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:02:53.773269 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:02:53.773361 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:02:53.774796 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.775566 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.776994 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:02:53.777394 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:02:53.777494 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:02:53.781974 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:02:53.781987 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:02:53.782003 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:02:53.782034 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:02:53.784705 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.785394 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.785401 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.785404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.785538 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:02:53.785973 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:02:53.785976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:02:53.785978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:02:53.786369 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:02:53.786572 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 01:02:53.786870 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 01:02:53.787032 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:02:53.788002 23163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:02:53.815567 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.815050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.0.0.1:44534)
0806 01:02:53.815248 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.815334 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:35571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 01:02:53.816555 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:53.816187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.0.0.1:44548)
0806 01:02:53.816361 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:53.816452 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:35571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 01:02:53.876850 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 22994
I20260806 01:02:53.902892 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 22464
I20260806 01:02:53.904214 22546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:02:54.018561 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:02:54.017718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:35571 (local address 127.0.0.1:44558)
0806 01:02:54.018274 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 01:02:54.018364 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:35571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360}
I20260806 01:02:54.018716 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 22464
I20260806 01:02:54.057451 5554 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 01:02:54.402002 5554 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 01:02:54.402282 5554 mini_ranger.cc:66] Stopping Ranger...
I20260806 01:02:54.756879 5554 mini_ranger.cc:68] Stopped Ranger
2026-08-06 01:02:54.757 UTC [21886] LOG: received smart shutdown request
2026-08-06 01:02:54.761 UTC [21886] LOG: background worker "logical replication launcher" (PID 21897) exited with exit code 1
2026-08-06 01:02:54.761 UTC [21890] LOG: shutting down
2026-08-06 01:02:54.909 UTC [21886] LOG: database system is shut down
2026-08-06T01:02:54Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50734 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:02:55 dist-test-slave-x4qh krb5kdc[23255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:02:55 dist-test-slave-x4qh krb5kdc[23255](info): set up 2 sockets
Aug 06 01:02:55 dist-test-slave-x4qh krb5kdc[23255](info): commencing operation
krb5kdc: starting...
W20260806 01:02:56.310585 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:02:56 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:02:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:02:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.108.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.108.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/rangeradmin_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/rangeradmin_127.5.108.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.108.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.108.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/rangerlookup_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/rangerlookup_127.5.108.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.108.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.108.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/HTTP_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/HTTP_127.5.108.148@KRBTEST.COM.keytab.
I20260806 01:02:56.694278 5554 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskq1gvHq/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/postgres -l logfile start
2026-08-06 01:02:58.936 UTC [23298] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 01:02:58.936 UTC [23298] LOG: listening on IPv4 address "127.5.108.148", port 48817
2026-08-06 01:02:58.939 UTC [23298] LOG: listening on Unix socket "/tmp/.s.PGSQL.48817"
2026-08-06 01:02:58.961 UTC [23302] LOG: database system was shut down at 2026-08-06 01:02:57 UTC
2026-08-06 01:02:58.965 UTC [23298] LOG: database system is ready to accept connections
I20260806 01:03:00.093567 5554 mini_postgres.cc:96] Postgres bound to 48817
2026-08-06 01:03:00.117 UTC [23312] FATAL: database "slave" does not exist
127.5.108.148:48817 - accepting connections
I20260806 01:03:00.118088 5554 mini_ranger.cc:161] Starting Ranger...
I20260806 01:03:00.145866 5554 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 01:03:00.572572 5554 mini_ranger.cc:87] Created ranger Postgres database
I20260806 01:03:00.572710 5554 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-admin
2026-08-06 01:03:00,849 [I] DB FLAVOR :POSTGRES
2026-08-06 01:03:00,850 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:03:00,850 [I] Checking connection..
2026-08-06 01:03:00,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 01:03:01,091 [I] Checking connection passed.
2026-08-06 01:03:01,091 [I] --------- Verifying version history table ---------
2026-08-06 01:03:01,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:03:01,345 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:03:01,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:03:01,595 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:03:01,596 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 01:03:01,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 01:03:01.825 UTC [23405] WARNING: there is no transaction in progress
2026-08-06 01:03:01,834 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 01:03:01,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:03:02,077 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 01:03:02,078 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:03:02,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 01:03:02,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 01:03:02,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x4qh.c.gcp-upstream.internal','N') ;"
2026-08-06 01:03:02,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:03:03,023 [I] Table x_portal_user does not exist in database ranger
2026-08-06 01:03:03,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:03:03,265 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 01:03:03,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:03:03,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:03:03,750 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 01:03:03,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 01:03:04.650 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.665 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.680 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.693 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.708 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.724 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.773 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.780 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.792 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.801 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.810 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.823 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.832 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.838 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.844 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:04.850 UTC [23565] WARNING: there is no transaction in progress
2026-08-06 01:03:05,412 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 01:03:05,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 01:03:05,660 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 01:03:05,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:03:05,921 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 01:03:05,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:03:06,179 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 01:03:06,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:03:06,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:03:06,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x4qh.c.gcp-upstream.internal';"
2026-08-06 01:03:06,947 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 01:03:06,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 01:03:07,192 [I] DB_PATCHES have already been applied
I20260806 01:03:07.198423 5554 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-admin:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskq1gvHq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 01:03:07.198541 5554 mini_ranger.cc:193] Using host: 127.5.108.148
I20260806 01:03:07.219704 5554 mini_ranger.cc:227] Ranger admin URL: http://127.5.108.148:43039
Aug 06, 2026 1:03:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:03:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:03:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 1:03:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 1:03:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 1:03:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 1:03:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 01:03:08.534649 5554 mini_ranger.cc:160] Time spent starting Ranger: real 8.417s user 0.000s sys 0.053s
Aug 06, 2026 1:03:27 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 1:03:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 1:03:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 1:03:27 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 1:03:28 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 01:03:28.994397 5554 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.108.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.108.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/rangerkms_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.108.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/rangerkms_127.5.108.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.108.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.108.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.108.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/HTTP_127.5.108.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.108.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/HTTP_127.5.108.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 01:03:29 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978209, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 01:03:29.193164 5554 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 01:03:29.218722 5554 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 01:03:29.959981 5554 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 01:03:29.960105 5554 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-kms
I20260806 01:03:29.960155 5554 mini_ranger_kms.cc:227] Using postgres at 127.5.108.148:48817
2026-08-06 01:03:30,360 [I] DB FLAVOR :POSTGRES
2026-08-06 01:03:30,361 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:03:30,361 [I] Checking connection
2026-08-06 01:03:30,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 01:03:30,627 [I] connection success
2026-08-06 01:03:30,627 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 01:03:30,627 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 01:03:30,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 01:03:30,913 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 01:03:30,913 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:03:30,914 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 01:03:30,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.108.148:48817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 01:03:31,167 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 01:03:31.650771 5554 mini_ranger_kms.cc:310] Created kms service
I20260806 01:03:31.857698 5554 mini_ranger_kms.cc:326] Created kudu user
I20260806 01:03:31.902853 5554 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 01:03:31 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978209, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:32.295617 5554 mini_ranger_kms.cc:384] Added ranger policy
I20260806 01:03:32.295961 5554 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-kms:/tmp/dist-test-taskq1gvHq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskq1gvHq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 01:03:32.296029 5554 mini_ranger_kms.cc:240] Using host: 127.5.108.148
I20260806 01:03:32.319047 5554 mini_ranger_kms.cc:276] Ranger KMS PID: 23819
I20260806 01:03:32.319159 5554 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.108.148:52247
Aug 06, 2026 1:03:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:03:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:03:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 1:03:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 1:03:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 1:03:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 01:03:34 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978214, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.108.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 01:03:35 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978214, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.108.148@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:35.042778 5554 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.850s user 0.070s sys 0.361s
I20260806 01:03:35.043020 5554 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 01:03:35.043195 5554 mini_ranger_kms.cc:401] 127.5.108.148:52247/kms/v1/keys
Aug 06 01:03:35 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978214, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.108.148@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
Aug 06 01:03:37 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:03:37.762924 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44197
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:44197
--ranger_config_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:52247/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:03:38.095757 23885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:38.095988 23885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:38.096124 23885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:38.102679 23885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:03:38.102778 23885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:38.102821 23885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:38.102905 23885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:03:38.102986 23885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:03:38.112311 23885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.108.148:52247/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:44197
--ranger_config_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:38.113385 23885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:38.114528 23885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:38.126235 23891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:38.126175 23893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:38.126276 23895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:38.126395 23885 server_base.cc:1034] running on GCE node
I20260806 01:03:38.127630 23885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:38.129036 23885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:38.130264 23885 hybrid_clock.cc:648] HybridClock initialized: now 1785978218130214 us; error 44 us; skew 500 ppm
I20260806 01:03:38.130628 23885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:38 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978218, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:38.133744 23885 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:03:38.135044 23885 webserver.cc:460] Webserver started at http://127.5.108.190:37017/ using document root <none> and password file <none>
I20260806 01:03:38.135681 23885 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:38.135778 23885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:38.136015 23885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:03:38 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978218, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:38.193135 23885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "0718165bf7174f3181afdcbce8e596db"
format_stamp: "Formatted at 2026-08-06 01:03:38 on dist-test-slave-x4qh"
server_key: "d261bae5abad81ba1f3994d93efc056d"
server_key_iv: "29f456aa0b29dca4f8ce21b864bd9668"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:38.194224 23885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "0718165bf7174f3181afdcbce8e596db"
format_stamp: "Formatted at 2026-08-06 01:03:38 on dist-test-slave-x4qh"
server_key: "d261bae5abad81ba1f3994d93efc056d"
server_key_iv: "29f456aa0b29dca4f8ce21b864bd9668"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:38.201002 23885 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 01:03:38.219965 23904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:38.221715 23885 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 01:03:38.221909 23885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "0718165bf7174f3181afdcbce8e596db"
format_stamp: "Formatted at 2026-08-06 01:03:38 on dist-test-slave-x4qh"
server_key: "d261bae5abad81ba1f3994d93efc056d"
server_key_iv: "29f456aa0b29dca4f8ce21b864bd9668"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:38.222154 23885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:38.252017 23885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:38.256480 23885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:38.256812 23885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:38.276805 23977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:44197 every 8 connection(s)
I20260806 01:03:38.276815 23885 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:44197
I20260806 01:03:38.278541 23885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:03:38.281222 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 23885
I20260806 01:03:38.281443 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:03:38.282374 5554 external_mini_cluster.cc:1428] Setting key f84b90cf8187ab903513bef314d62f47
I20260806 01:03:38.284503 23979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:38.302148 23979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Bootstrap starting.
I20260806 01:03:38.306015 23979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:38.307071 23979 log.cc:824] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:38.310134 23979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: No bootstrap required, opened a new log
Aug 06 01:03:38 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:38.320840 23979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } }
I20260806 01:03:38.321368 23979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:38.321496 23979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0718165bf7174f3181afdcbce8e596db, State: Initialized, Role: FOLLOWER
I20260806 01:03:38.322225 23979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } }
I20260806 01:03:38.322566 23979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:03:38.322793 23979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:03:38.323048 23979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:38.324610 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:38.284528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38642 (local address 127.5.108.190:44197)
0806 01:03:38.299982 (+ 15454us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:38.299997 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:38.300053 (+ 56us) server_negotiation.cc:407] Connection header received
0806 01:03:38.302490 (+ 2437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:38.302520 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:38.303006 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:38.303455 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:38.305305 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:38.307444 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:38.309047 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:38.310086 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:38.310286 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:38.315337 (+ 5051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:38.315348 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:38.315403 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:38.315477 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:38.319693 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:38.320613 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:38.320622 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:38.320631 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:38.320809 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:38.321270 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:38.321275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:38.321279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:38.321949 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:38.322243 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:38.322789 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 01:03:38.323188 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15186,"thread_start_us":14973,"threads_started":1}
I20260806 01:03:38.325687 23979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } }
I20260806 01:03:38.326186 23979 leader_election.cc:302] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0718165bf7174f3181afdcbce8e596db; no voters:
I20260806 01:03:38.326884 23979 leader_election.cc:288] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:03:38.327083 23985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:03:38.329126 23985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 1 LEADER]: Becoming Leader. State: Replica: 0718165bf7174f3181afdcbce8e596db, State: Running, Role: LEADER
I20260806 01:03:38.329783 23985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } }
I20260806 01:03:38.330192 23979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:03:38.332827 23988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0718165bf7174f3181afdcbce8e596db. Latest consensus state: current_term: 1 leader_uuid: "0718165bf7174f3181afdcbce8e596db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } } }
I20260806 01:03:38.333283 23988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:38.333313 23986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0718165bf7174f3181afdcbce8e596db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } } }
I20260806 01:03:38.333673 23986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:38.345690 23993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:03:38.355171 23993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:03:38.356027 23979 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 01:03:38.375851 23993 catalog_manager.cc:1347] Generated new cluster ID: f0ee00aaf8944861a236c8ad7a97e7e0
I20260806 01:03:38.376003 23993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:03:38.408072 23993 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:03:38.410101 23993 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:03:38.428771 23993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Generated new TSK 0
I20260806 01:03:38.429766 23993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:03:39.130941 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:52247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44197
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:03:39 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978219, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 01:03:39.533269 24046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:39.533577 24046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:39.533756 24046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:39.542727 24046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:39.542866 24046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:39.543069 24046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:03:39.555104 24046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.108.148:52247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:39.556427 24046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:39.557893 24046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:39.577350 24060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:39.586429 24061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:39.586740 24046 server_base.cc:1034] running on GCE node
W20260806 01:03:39.588346 24064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:39.589243 24046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:39.591665 24046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:39.592983 24046 hybrid_clock.cc:648] HybridClock initialized: now 1785978219592858 us; error 111 us; skew 500 ppm
I20260806 01:03:39.593382 24046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:39 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978219, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:39.597801 24046 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:03:39.599797 24046 webserver.cc:460] Webserver started at http://127.5.108.129:39427/ using document root <none> and password file <none>
I20260806 01:03:39.600736 24046 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:39.600873 24046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:39.601225 24046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:03:39 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978219, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:39.622911 24046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "f718192538c640faa96eb2cfa5d97a58"
format_stamp: "Formatted at 2026-08-06 01:03:39 on dist-test-slave-x4qh"
server_key: "b83b07eed228e6216a27d267e5d99f79"
server_key_iv: "0b1cb2294ffcff050dfb020eee917bb2"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:39.624095 24046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "f718192538c640faa96eb2cfa5d97a58"
format_stamp: "Formatted at 2026-08-06 01:03:39 on dist-test-slave-x4qh"
server_key: "b83b07eed228e6216a27d267e5d99f79"
server_key_iv: "0b1cb2294ffcff050dfb020eee917bb2"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:39.641116 24046 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.003s sys 0.012s
I20260806 01:03:39.680918 24075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:39.682938 24046 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.001s sys 0.009s
I20260806 01:03:39.683225 24046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "f718192538c640faa96eb2cfa5d97a58"
format_stamp: "Formatted at 2026-08-06 01:03:39 on dist-test-slave-x4qh"
server_key: "b83b07eed228e6216a27d267e5d99f79"
server_key_iv: "0b1cb2294ffcff050dfb020eee917bb2"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:39.683606 24046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:39.711599 24046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:39.716897 24046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:39.717278 24046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:39.718755 24046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:39.720974 24046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:39.721110 24046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:03:39.721325 24046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:39.721472 24046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:39.796415 24046 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:37345
I20260806 01:03:39.796684 24232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:37345 every 8 connection(s)
I20260806 01:03:39.798213 24046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:03:39.807051 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 24046
I20260806 01:03:39.807250 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:03:39.808115 5554 external_mini_cluster.cc:1428] Setting key 92112dc4f802cc0b400df84dcff3b553
Aug 06 01:03:39 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978219, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:39.879112 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:39.803589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:55097 (local address 127.5.108.190:44197)
0806 01:03:39.834465 (+ 30876us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:39.834473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:39.839171 (+ 4698us) server_negotiation.cc:407] Connection header received
0806 01:03:39.842728 (+ 3557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:39.842734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:39.842831 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:39.842933 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:39.862520 (+ 19587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:39.863599 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:39.867150 (+ 3551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:39.867750 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:39.867837 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:39.872093 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:39.872099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:39.872102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:39.872139 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:39.875252 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:39.876148 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:39.876153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:39.876156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:39.876297 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:39.876899 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:39.876904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:39.876907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:39.877288 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:39.877437 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:39.878445 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0806 01:03:39.878705 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30745,"thread_start_us":39451,"threads_started":1}
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
I20260806 01:03:39.881570 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:39.801744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44197 (local address 127.5.108.129:55097)
0806 01:03:39.836696 (+ 34952us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:39.836760 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:39.841724 (+ 4964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:39.858255 (+ 16531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:39.858273 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:39.859048 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:39.860065 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:39.860092 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:39.863814 (+ 3722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:39.863819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:39.866999 (+ 3180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:39.867013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:39.868172 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:39.868179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:39.868442 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:39.869230 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:39.869257 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:39.871838 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:39.875456 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:39.875465 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:39.875480 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:39.876017 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:39.876431 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:39.876434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:39.876437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:39.876728 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:39.877515 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:39.877523 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:39.877970 (+ 447us) client_negotiation.cc:769] Sending connection context
0806 01:03:39.879472 (+ 1502us) client_negotiation.cc:240] Negotiation successful
0806 01:03:39.880000 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31947,"thread_start_us":32624,"threads_started":1}
I20260806 01:03:39.883637 24235 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44197
I20260806 01:03:39.883970 24235 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:39.884852 24235 heartbeater.cc:511] Master 127.5.108.190:44197 requested a full tablet report, sending...
I20260806 01:03:39.887650 23929 ts_manager.cc:194] Registered new tserver with Master: f718192538c640faa96eb2cfa5d97a58 (127.5.108.129:37345)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
I20260806 01:03:39.889820 23929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:55097
Aug 06 01:03:39 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978219, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:03:39.936983 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:52247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44197
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:03:40 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978219, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
W20260806 01:03:40.405580 24242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:40.406055 24242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:40.406257 24242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:40.417582 24242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:40.417811 24242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:40.418041 24242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:03:40.433247 24242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.108.148:52247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:40.435055 24242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:40.436803 24242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:40.457926 24251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:40.459491 24252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:40.468019 24256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:40.468709 24242 server_base.cc:1034] running on GCE node
I20260806 01:03:40.469435 24242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:40.471655 24242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:40.472965 24242 hybrid_clock.cc:648] HybridClock initialized: now 1785978220472857 us; error 85 us; skew 500 ppm
I20260806 01:03:40.473323 24242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:40 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978220, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:40.477317 24242 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:03:40.479471 24242 webserver.cc:460] Webserver started at http://127.5.108.130:35237/ using document root <none> and password file <none>
I20260806 01:03:40.480376 24242 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:40.480526 24242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:40.480911 24242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:03:40 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978220, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:40.498735 24242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "b34bc0f5630145288df2da46665f0cfe"
format_stamp: "Formatted at 2026-08-06 01:03:40 on dist-test-slave-x4qh"
server_key: "8b4c4f6928802f134fdaade1be09ae37"
server_key_iv: "5f0fb276f520c262630f65a44e4d413a"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:40.499854 24242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "b34bc0f5630145288df2da46665f0cfe"
format_stamp: "Formatted at 2026-08-06 01:03:40 on dist-test-slave-x4qh"
server_key: "8b4c4f6928802f134fdaade1be09ae37"
server_key_iv: "5f0fb276f520c262630f65a44e4d413a"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:40.508584 24242 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260806 01:03:40.533135 24271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:40.535148 24242 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.007s sys 0.000s
I20260806 01:03:40.535454 24242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "b34bc0f5630145288df2da46665f0cfe"
format_stamp: "Formatted at 2026-08-06 01:03:40 on dist-test-slave-x4qh"
server_key: "8b4c4f6928802f134fdaade1be09ae37"
server_key_iv: "5f0fb276f520c262630f65a44e4d413a"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:40.535805 24242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:40.557171 24242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:40.562450 24242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:40.562831 24242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:40.564122 24242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:40.565984 24242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:40.566104 24242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:40.566263 24242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:40.566363 24242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:40.616529 24242 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:44319
I20260806 01:03:40.616573 24431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:44319 every 8 connection(s)
I20260806 01:03:40.618321 24242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:03:40.622598 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 24242
I20260806 01:03:40.622764 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:03:40.623332 5554 external_mini_cluster.cc:1428] Setting key a166654302aa053965f087cb9423841d
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Aug 06 01:03:40 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978220, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:40.676743 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:40.624070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:33161 (local address 127.5.108.190:44197)
0806 01:03:40.646876 (+ 22806us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:40.646883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:40.648128 (+ 1245us) server_negotiation.cc:407] Connection header received
0806 01:03:40.651742 (+ 3614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:40.651750 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:40.651882 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:40.651983 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:40.654320 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:40.655483 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:40.659414 (+ 3931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:40.660075 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:40.660238 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:40.668644 (+ 8406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:40.668663 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:40.668671 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:40.668724 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:40.672784 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:40.673927 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:40.673932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:40.673936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:40.674085 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:40.674662 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:40.674667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:40.674670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:40.674975 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:40.675132 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:40.676114 (+ 982us) server_negotiation.cc:299] Negotiation successful
0806 01:03:40.676345 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22662,"thread_start_us":22818,"threads_started":1}
I20260806 01:03:40.678015 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:40.621763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44197 (local address 127.5.108.130:33161)
0806 01:03:40.647593 (+ 25830us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:40.647667 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:40.650439 (+ 2772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:40.652550 (+ 2111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:40.652563 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:40.653264 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:40.654085 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:40.654111 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:40.655723 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:40.655729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:40.659147 (+ 3418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:40.659164 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:40.660356 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:40.660363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:40.660672 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:40.661482 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:40.661530 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:40.668269 (+ 6739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:40.672980 (+ 4711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:40.673009 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:40.673031 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:40.673744 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:40.674224 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:40.674230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:40.674235 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:40.674539 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:40.675121 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:40.675129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:40.675662 (+ 533us) client_negotiation.cc:769] Sending connection context
0806 01:03:40.676033 (+ 371us) client_negotiation.cc:240] Negotiation successful
0806 01:03:40.676553 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25475,"thread_start_us":28887,"threads_started":1}
I20260806 01:03:40.679943 24432 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44197
I20260806 01:03:40.680292 24432 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:40.681185 24432 heartbeater.cc:511] Master 127.5.108.190:44197 requested a full tablet report, sending...
I20260806 01:03:40.682984 23929 ts_manager.cc:194] Registered new tserver with Master: b34bc0f5630145288df2da46665f0cfe (127.5.108.130:44319)
I20260806 01:03:40.684238 23929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:33161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:03:40.700737 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:52247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44197
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:03:40.894523 24235 heartbeater.cc:503] Master 127.5.108.190:44197 was elected leader, sending a full tablet report...
W20260806 01:03:41.044220 24441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:41.044498 24441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:41.044660 24441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:41.051645 24441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:41.051738 24441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:41.051841 24441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:03:41.060600 24441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.108.148:52247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:41.061672 24441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:41.062858 24441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:41.075778 24448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:41.075894 24450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:41.075902 24446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:41.076858 24441 server_base.cc:1034] running on GCE node
I20260806 01:03:41.077471 24441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:41.078382 24441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:41.079627 24441 hybrid_clock.cc:648] HybridClock initialized: now 1785978221079552 us; error 54 us; skew 500 ppm
I20260806 01:03:41.079989 24441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:41 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978221, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:41.083089 24441 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:03:41.084542 24441 webserver.cc:460] Webserver started at http://127.5.108.131:36797/ using document root <none> and password file <none>
I20260806 01:03:41.085136 24441 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:41.085245 24441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:41.085564 24441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:03:41 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978221, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:41.101554 24441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "677c003585d649d6928b7a74bb466b7a"
format_stamp: "Formatted at 2026-08-06 01:03:41 on dist-test-slave-x4qh"
server_key: "3a6f9cc72f4c96c9d618d54af4b0b2b7"
server_key_iv: "739f05580ef0e54f2b0e304a0479b94a"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:41.102357 24441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "677c003585d649d6928b7a74bb466b7a"
format_stamp: "Formatted at 2026-08-06 01:03:41 on dist-test-slave-x4qh"
server_key: "3a6f9cc72f4c96c9d618d54af4b0b2b7"
server_key_iv: "739f05580ef0e54f2b0e304a0479b94a"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:41.107750 24441 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 01:03:41.122804 24458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:41.124245 24441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:03:41.124434 24441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "677c003585d649d6928b7a74bb466b7a"
format_stamp: "Formatted at 2026-08-06 01:03:41 on dist-test-slave-x4qh"
server_key: "3a6f9cc72f4c96c9d618d54af4b0b2b7"
server_key_iv: "739f05580ef0e54f2b0e304a0479b94a"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:41.124636 24441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:41.141714 24441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:41.145371 24441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:41.145706 24441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:41.146696 24441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:41.148080 24441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:41.148155 24441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:41.148334 24441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:41.148387 24441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:41.181828 24441 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:34473
I20260806 01:03:41.181864 24610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:34473 every 8 connection(s)
I20260806 01:03:41.183149 24441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:03:41.187119 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 24441
I20260806 01:03:41.187304 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:03:41.187832 5554 external_mini_cluster.cc:1428] Setting key 1045b6ed0566bce3fc32ff60de9a989d
Aug 06 01:03:41 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978221, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:41.203980 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:41.185660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:36551 (local address 127.5.108.190:44197)
0806 01:03:41.186175 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:41.186179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:41.187069 (+ 890us) server_negotiation.cc:407] Connection header received
0806 01:03:41.189711 (+ 2642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:41.189739 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:41.189829 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:41.189945 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:41.191765 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.192508 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.193988 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.194400 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.194486 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:41.197976 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:41.197980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:41.197984 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:41.198014 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:41.200991 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:41.201769 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:41.201773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:41.201776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:41.201936 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:41.202454 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:41.202459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:41.202463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:41.202776 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:41.202920 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:41.203581 (+ 661us) server_negotiation.cc:299] Negotiation successful
0806 01:03:41.203750 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":255,"threads_started":1}
I20260806 01:03:41.204833 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:41.186218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44197 (local address 127.5.108.131:36551)
0806 01:03:41.186917 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:41.186953 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:41.189041 (+ 2088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:41.190273 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:41.190284 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:41.190914 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:41.191615 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:41.191633 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.192636 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.192640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:41.193868 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:41.193879 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.194572 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.194576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:41.194765 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:41.195444 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:41.195468 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:41.197775 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:41.201125 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:41.201132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:41.201159 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:41.201653 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:41.202082 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:41.202090 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:41.202096 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:41.202314 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:41.202900 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:41.202905 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:41.203227 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 01:03:41.203481 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 01:03:41.203802 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":300,"threads_started":1}
I20260806 01:03:41.206151 24613 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44197
I20260806 01:03:41.206435 24613 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:41.207132 24613 heartbeater.cc:511] Master 127.5.108.190:44197 requested a full tablet report, sending...
I20260806 01:03:41.208539 23929 ts_manager.cc:194] Registered new tserver with Master: 677c003585d649d6928b7a74bb466b7a (127.5.108.131:34473)
I20260806 01:03:41.209321 23929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:36551
I20260806 01:03:41.215440 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:03:41.217127 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 24046
Aug 06 01:03:41 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:03:41.233242 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:41.218094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50546 (local address 127.5.108.129:37345)
0806 01:03:41.218711 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:41.218716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:41.218738 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:03:41.218792 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:41.218797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:41.218962 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:41.219076 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:41.220503 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.221475 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.222825 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.223320 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.223539 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:41.227538 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:41.227548 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:41.227558 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:41.227587 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:41.230640 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:41.231432 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:41.231439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:41.231441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:41.231610 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:41.231931 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:41.231934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:41.231936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:41.232220 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:41.232440 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:41.232665 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 01:03:41.232861 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":334,"threads_started":1}
I20260806 01:03:41.234382 24224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:41.320223 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 24046
I20260806 01:03:41.343778 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 24242
Aug 06 01:03:41 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:03:41.359189 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:41.344762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56528 (local address 127.5.108.130:44319)
0806 01:03:41.345388 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:41.345396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:41.345422 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:03:41.345495 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:41.345501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:41.345705 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:41.345838 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:41.347016 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.348103 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.349445 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.349981 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.350083 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:41.354189 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:41.354199 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:41.354212 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:41.354253 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:41.356817 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:41.357350 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:41.357356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:41.357358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:41.357484 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:41.357813 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:41.357816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:41.357818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:41.358136 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:41.358346 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:41.358582 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 01:03:41.358822 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":329,"threads_started":1}
I20260806 01:03:41.360488 24426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:41.438756 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 24242
I20260806 01:03:41.465895 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 24441
Aug 06 01:03:41 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:03:41.482359 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:41.466954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45886 (local address 127.5.108.131:34473)
0806 01:03:41.467642 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:41.467649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:41.467680 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:03:41.467756 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:41.467762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:41.467974 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:41.468144 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:41.469562 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.470561 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.471915 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:41.472438 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:41.472588 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:41.476964 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:41.476974 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:41.476986 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:41.477022 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:41.479666 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:41.480308 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:41.480315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:41.480317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:41.480436 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:41.480834 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:41.480839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:41.480842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:41.481158 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:41.481475 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:41.481755 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 01:03:41.482007 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":338,"threads_started":1}
I20260806 01:03:41.483784 24606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:41.574618 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 24441
I20260806 01:03:41.597410 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 23885
I20260806 01:03:41.598706 23971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:41.700179 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 23885
I20260806 01:03:41.734697 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44197
--webserver_interface=127.5.108.190
--webserver_port=37017
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:44197
--ranger_config_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:52247/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.108.148:52247/kms with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:03:42.074011 24629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:42.074306 24629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:42.074397 24629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:42.081863 24629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:03:42.081972 24629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:42.082026 24629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:42.082086 24629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:03:42.082137 24629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:03:42.092306 24629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.108.148:52247/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:44197
--ranger_config_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=37017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:42.093582 24629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:42.094828 24629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:42.106385 24639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:42.106482 24637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:42.106503 24636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:42.107256 24629 server_base.cc:1034] running on GCE node
I20260806 01:03:42.108042 24629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:42.109347 24629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:42.110544 24629 hybrid_clock.cc:648] HybridClock initialized: now 1785978222110496 us; error 42 us; skew 500 ppm
I20260806 01:03:42.110910 24629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:42 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978222, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:42.119029 24629 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:03:42.120312 24629 webserver.cc:460] Webserver started at http://127.5.108.190:37017/ using document root <none> and password file <none>
I20260806 01:03:42.120925 24629 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:42.121066 24629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:03:42 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978222, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:42.168701 24629 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 01:03:42.171749 24649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:42.172993 24629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:03:42.173177 24629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "0718165bf7174f3181afdcbce8e596db"
format_stamp: "Formatted at 2026-08-06 01:03:38 on dist-test-slave-x4qh"
server_key: "d261bae5abad81ba1f3994d93efc056d"
server_key_iv: "29f456aa0b29dca4f8ce21b864bd9668"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:42.174134 24629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:42.188524 24629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:42.191802 24629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:42.192072 24629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:42.210394 24629 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:44197
I20260806 01:03:42.210428 24726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:44197 every 8 connection(s)
I20260806 01:03:42.211777 24629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:03:42.212697 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 24629
I20260806 01:03:42.213590 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:52247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:37345
--local_ip_for_outbound_sockets=127.5.108.129
--tserver_master_addrs=127.5.108.190:44197
--webserver_port=39427
--webserver_interface=127.5.108.129
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.108.148:52247/kms with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:03:42.218538 24727 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:03:42.221704 24727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Bootstrap starting.
I20260806 01:03:42.246270 24727 log.cc:824] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:42.257920 24727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:03:42.258591 24727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Bootstrap complete.
I20260806 01:03:42.272363 24727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } }
I20260806 01:03:42.273649 24727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0718165bf7174f3181afdcbce8e596db, State: Initialized, Role: FOLLOWER
I20260806 01:03:42.274407 24727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } }
I20260806 01:03:42.274698 24727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:03:42.274830 24727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:03:42.275024 24727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:03:42.281608 24727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } }
I20260806 01:03:42.282258 24727 leader_election.cc:302] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0718165bf7174f3181afdcbce8e596db; no voters:
I20260806 01:03:42.282903 24727 leader_election.cc:288] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:03:42.283047 24731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:03:42.284380 24731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [term 2 LEADER]: Becoming Leader. State: Replica: 0718165bf7174f3181afdcbce8e596db, State: Running, Role: LEADER
I20260806 01:03:42.284994 24731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } }
I20260806 01:03:42.285213 24727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:03:42.287635 24734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0718165bf7174f3181afdcbce8e596db. Latest consensus state: current_term: 2 leader_uuid: "0718165bf7174f3181afdcbce8e596db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } } }
I20260806 01:03:42.288021 24734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:42.288563 24732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0718165bf7174f3181afdcbce8e596db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718165bf7174f3181afdcbce8e596db" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44197 } } }
I20260806 01:03:42.288851 24732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:42.297967 24738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:03:42.303828 24738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:03:42.305300 24738 catalog_manager.cc:1259] Loaded cluster ID: f0ee00aaf8944861a236c8ad7a97e7e0
I20260806 01:03:42.305449 24738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:03:42.309635 24738 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:03:42.311136 24738 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Loaded TSK: 0
I20260806 01:03:42.312697 24738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:03:42.321539 24727 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260806 01:03:42.694150 24729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:42.694630 24729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:42.694868 24729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:42.706481 24729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:42.706616 24729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:42.706811 24729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:03:42.721835 24729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.108.148:52247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:37345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=39427
--tserver_master_addrs=127.5.108.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:42.723668 24729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:42.725545 24729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:42.752422 24772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:42.752435 24773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:42.752935 24775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:42.753628 24729 server_base.cc:1034] running on GCE node
I20260806 01:03:42.754288 24729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:42.755651 24729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:42.756943 24729 hybrid_clock.cc:648] HybridClock initialized: now 1785978222756887 us; error 51 us; skew 500 ppm
I20260806 01:03:42.757292 24729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:42 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978222, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:42.760667 24729 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:03:42.764150 24729 webserver.cc:460] Webserver started at http://127.5.108.129:39427/ using document root <none> and password file <none>
I20260806 01:03:42.765131 24729 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:42.765314 24729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:03:42 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978222, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:42.794052 24729 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:03:42.797078 24786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:42.798527 24729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:03:42.798707 24729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "f718192538c640faa96eb2cfa5d97a58"
format_stamp: "Formatted at 2026-08-06 01:03:39 on dist-test-slave-x4qh"
server_key: "b83b07eed228e6216a27d267e5d99f79"
server_key_iv: "0b1cb2294ffcff050dfb020eee917bb2"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:42.799942 24729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:42.829066 24729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:42.834134 24729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:42.834494 24729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:42.841643 24729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:42.849498 24729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:42.849637 24729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:42.849786 24729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:42.849891 24729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:42.913621 24729 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:37345
I20260806 01:03:42.913676 24943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:37345 every 8 connection(s)
I20260806 01:03:42.915206 24729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:03:42.924118 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 24729
I20260806 01:03:42.925109 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:52247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:44319
--local_ip_for_outbound_sockets=127.5.108.130
--tserver_master_addrs=127.5.108.190:44197
--webserver_port=35237
--webserver_interface=127.5.108.130
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.108.148:52247/kms with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:03:42 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978222, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:42.948694 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:42.919133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44197 (local address 127.5.108.129:58519)
0806 01:03:42.924020 (+ 4887us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:42.924085 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:42.927288 (+ 3203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:42.930246 (+ 2958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:42.930255 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:42.930800 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:42.931509 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:42.931535 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:42.933725 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:42.933730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:42.935700 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:42.935716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:42.937161 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:42.937168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:42.937425 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:42.938172 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:42.938194 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:42.940567 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:42.944205 (+ 3638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:42.944213 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:42.944226 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:42.944656 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:42.945143 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:42.945148 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:42.945151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:42.945334 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:42.946252 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:42.946259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:42.946541 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 01:03:42.946782 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 01:03:42.947314 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4519,"thread_start_us":15877,"threads_started":1}
I20260806 01:03:42.948681 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:42.919111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:58519 (local address 127.5.108.190:44197)
0806 01:03:42.927880 (+ 8769us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:42.927903 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:42.927945 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:03:42.928636 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:42.928655 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:42.929126 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:42.929540 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:42.931740 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:42.933496 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:42.935883 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:42.936709 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:42.936807 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:42.940836 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:42.940847 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:42.940866 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:42.940917 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:42.944058 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:42.944828 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:42.944834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:42.944839 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:42.945012 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:42.945454 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:42.945460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:42.945463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:42.946095 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:42.946299 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:42.947026 (+ 727us) server_negotiation.cc:299] Negotiation successful
0806 01:03:42.947405 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8501,"thread_start_us":8245,"threads_started":1}
I20260806 01:03:42.952682 24945 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44197
I20260806 01:03:42.952987 24945 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:42.953707 24945 heartbeater.cc:511] Master 127.5.108.190:44197 requested a full tablet report, sending...
W20260806 01:03:42.957583 24945 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:44197 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:03:42.957815 24945 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:42.958096 24945 heartbeater.cc:511] Master 127.5.108.190:44197 requested a full tablet report, sending...
I20260806 01:03:42.959506 24945 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:42.959769 24945 heartbeater.cc:511] Master 127.5.108.190:44197 requested a full tablet report, sending...
W20260806 01:03:42.961126 24945 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:03:43.337368 24950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:43.337739 24950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:43.337962 24950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:43.349246 24950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:43.349424 24950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:43.349649 24950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:03:43.364828 24950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.108.148:52247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:44319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=35237
--tserver_master_addrs=127.5.108.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:43.366351 24950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:43.367995 24950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:43.384531 24966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:43.385524 24967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:43.392973 24969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:43.393563 24950 server_base.cc:1034] running on GCE node
I20260806 01:03:43.394145 24950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:43.395103 24950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:43.396337 24950 hybrid_clock.cc:648] HybridClock initialized: now 1785978223396266 us; error 61 us; skew 500 ppm
I20260806 01:03:43.396632 24950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:43 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:43.400070 24950 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:03:43.405242 24950 webserver.cc:460] Webserver started at http://127.5.108.130:35237/ using document root <none> and password file <none>
I20260806 01:03:43.405970 24950 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:43.406132 24950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:03:43 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:43.444290 24950 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.000s
I20260806 01:03:43.463949 24989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:43.466003 24950 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.009s sys 0.011s
I20260806 01:03:43.466249 24950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "b34bc0f5630145288df2da46665f0cfe"
format_stamp: "Formatted at 2026-08-06 01:03:40 on dist-test-slave-x4qh"
server_key: "8b4c4f6928802f134fdaade1be09ae37"
server_key_iv: "5f0fb276f520c262630f65a44e4d413a"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:43.467806 24950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:43.552495 24950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:43.558363 24950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
Aug 06 01:03:43 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:43.558679 24950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:43.560510 24950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:43.584292 24950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:43.584491 24950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:43.584676 24950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:43.584769 24950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:43.653914 24950 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:44319
I20260806 01:03:43.653996 25138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:44319 every 8 connection(s)
I20260806 01:03:43.655817 24950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:03:43.664851 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 24950
I20260806 01:03:43.665663 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.108.148:52247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:34473
--local_ip_for_outbound_sockets=127.5.108.131
--tserver_master_addrs=127.5.108.190:44197
--webserver_port=36797
--webserver_interface=127.5.108.131
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.108.148:52247/kms with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:03:43 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:43.704882 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:43.659562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:60201 (local address 127.5.108.190:44197)
0806 01:03:43.683723 (+ 24161us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:43.683730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:43.683754 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:03:43.685158 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:43.685166 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:43.685272 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:43.685400 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:43.687543 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:43.688713 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:43.691149 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:43.691811 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:43.691946 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:43.696739 (+ 4793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:43.696746 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:43.696750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:43.696790 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:43.700610 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:43.701564 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:43.701569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:43.701573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:43.701712 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:43.702258 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:43.702263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:43.702267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:43.702661 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:43.702813 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:43.704262 (+ 1449us) server_negotiation.cc:299] Negotiation successful
0806 01:03:43.704500 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24024,"thread_start_us":23920,"threads_started":1}
I20260806 01:03:43.705941 25145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:43.660181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44197 (local address 127.5.108.130:60201)
0806 01:03:43.681767 (+ 21586us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:43.681829 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:43.684829 (+ 3000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:43.685740 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:43.685753 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:43.686410 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:43.687303 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:43.687330 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:43.688926 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:43.688932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:43.690952 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:43.690967 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:43.692290 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:43.692299 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:43.692576 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:43.693362 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:43.693392 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:43.696475 (+ 3083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:43.700782 (+ 4307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:43.700790 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:43.700805 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:43.701413 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:43.701851 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:43.701856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:43.701860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:43.702105 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:43.702922 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:43.702931 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:43.703354 (+ 423us) client_negotiation.cc:769] Sending connection context
0806 01:03:43.703693 (+ 339us) client_negotiation.cc:240] Negotiation successful
0806 01:03:43.704249 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21225,"thread_start_us":21009,"threads_started":1}
I20260806 01:03:43.707636 25140 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44197
I20260806 01:03:43.707988 25140 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:43.708818 25140 heartbeater.cc:511] Master 127.5.108.190:44197 requested a full tablet report, sending...
I20260806 01:03:43.710789 24677 ts_manager.cc:194] Registered new tserver with Master: b34bc0f5630145288df2da46665f0cfe (127.5.108.130:44319)
I20260806 01:03:43.713059 24677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:60201
I20260806 01:03:43.961344 24945 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:43.961895 24945 heartbeater.cc:511] Master 127.5.108.190:44197 requested a full tablet report, sending...
I20260806 01:03:43.963488 24677 ts_manager.cc:194] Registered new tserver with Master: f718192538c640faa96eb2cfa5d97a58 (127.5.108.129:37345)
I20260806 01:03:43.965060 24677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:58519
Aug 06 01:03:44 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
W20260806 01:03:44.132059 25147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:44.132421 25147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:44.132543 25147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:44.142246 25147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:44.142371 25147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:44.142575 25147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:03:44.153012 25147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.108.148:52247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:34473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=36797
--tserver_master_addrs=127.5.108.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:44.154229 25147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:44.155570 25147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:44.171607 25155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:44.171821 25154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:44.184809 25158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:44.185417 25147 server_base.cc:1034] running on GCE node
I20260806 01:03:44.186180 25147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:44.187427 25147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:44.188804 25147 hybrid_clock.cc:648] HybridClock initialized: now 1785978224188731 us; error 62 us; skew 500 ppm
I20260806 01:03:44.189208 25147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:44 dist-test-slave-x4qh krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978224, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:44.193133 25147 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:03:44.194880 25147 webserver.cc:460] Webserver started at http://127.5.108.131:36797/ using document root <none> and password file <none>
I20260806 01:03:44.195824 25147 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:44.195988 25147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:03:44 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978224, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:44.222375 25147 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:03:44.225792 25170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:44.227458 25147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:03:44.227633 25147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "677c003585d649d6928b7a74bb466b7a"
format_stamp: "Formatted at 2026-08-06 01:03:41 on dist-test-slave-x4qh"
server_key: "3a6f9cc72f4c96c9d618d54af4b0b2b7"
server_key_iv: "739f05580ef0e54f2b0e304a0479b94a"
server_key_version: "kuduclusterkey@0"
I20260806 01:03:44.229000 25147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:44.250866 25147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:44.255895 25147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:44.256269 25147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:44.258412 25147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:44.260147 25147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:44.260260 25147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:44.260365 25147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:44.260437 25147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:44.334327 25147 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:34473
I20260806 01:03:44.334698 25326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:34473 every 8 connection(s)
I20260806 01:03:44.336261 25147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:03:44.337227 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 25147
Aug 06 01:03:44 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for HTTP/127.5.108.148@KRBTEST.COM
I20260806 01:03:44.392231 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.338393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.5.108.190:44197)
0806 01:03:44.353652 (+ 15259us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.353659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.358198 (+ 4539us) server_negotiation.cc:407] Connection header received
0806 01:03:44.358423 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.358429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.358515 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.358772 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:44.361087 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.362240 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.371148 (+ 8908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.371873 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.371996 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.379992 (+ 7996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:44.379998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:44.380002 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:44.380081 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:44.383797 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:44.386216 (+ 2419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:44.386223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:44.386226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:44.386404 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:44.390450 (+ 4046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:44.390455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:44.390458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:44.390832 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:44.391073 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:44.391664 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 01:03:44.391882 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15109,"thread_start_us":15737,"threads_started":1}
Aug 06 01:03:44 dist-test-slave-x4qh krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978224, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:44.415441 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.363643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:34959 (local address 127.5.108.190:44197)
0806 01:03:44.385417 (+ 21774us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.385423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.389933 (+ 4510us) server_negotiation.cc:407] Connection header received
0806 01:03:44.390000 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.390005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.390078 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.390183 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:44.395206 (+ 5023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.396467 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.398892 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.399555 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.399667 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.408354 (+ 8687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:44.408365 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:44.408373 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:44.408417 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:44.411839 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:44.412767 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:44.412772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:44.412776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:44.412916 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:44.413434 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:44.413438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:44.413441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:44.413789 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:44.413908 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:44.414868 (+ 960us) server_negotiation.cc:299] Negotiation successful
0806 01:03:44.415063 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21645,"thread_start_us":21782,"threads_started":1}
I20260806 01:03:44.417882 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.360359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44197 (local address 127.5.108.131:34959)
0806 01:03:44.386993 (+ 26634us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:44.387047 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:44.389536 (+ 2489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:44.393411 (+ 3875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:44.393424 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:44.394042 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:44.394969 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.394995 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.396683 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.396688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.398726 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.398748 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.402096 (+ 3348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.402108 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.402393 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.403254 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:44.403286 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:44.408068 (+ 4782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:44.412010 (+ 3942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:44.412018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:44.412035 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:44.412615 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:44.413062 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:44.413066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:44.413070 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:44.413298 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:44.414021 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:44.414028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:44.414432 (+ 404us) client_negotiation.cc:769] Sending connection context
0806 01:03:44.415941 (+ 1509us) client_negotiation.cc:240] Negotiation successful
0806 01:03:44.416487 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25439,"thread_start_us":25205,"threads_started":1}
I20260806 01:03:44.420301 25327 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44197
I20260806 01:03:44.420611 25327 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:44.421393 25327 heartbeater.cc:511] Master 127.5.108.190:44197 requested a full tablet report, sending...
I20260806 01:03:44.422859 24675 ts_manager.cc:194] Registered new tserver with Master: 677c003585d649d6928b7a74bb466b7a (127.5.108.131:34473)
I20260806 01:03:44.424119 24675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:34959
I20260806 01:03:44.433128 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:03:44.510330 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.477417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.5.108.190:44197)
0806 01:03:44.477636 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.477643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.492044 (+ 14401us) server_negotiation.cc:407] Connection header received
0806 01:03:44.492262 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.492268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.492378 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.492497 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:44.494480 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.495629 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.497706 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.498390 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.498514 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.502292 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:44.502299 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:44.502304 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:44.502346 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:44.506498 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:44.507399 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:44.507404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:44.507408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:44.507548 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:44.508012 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:44.508017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:44.508021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:44.508397 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:44.508619 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:44.509750 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0806 01:03:44.509984 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 01:03:44.526475 24675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:03:44.545612 24675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:03:44.614919 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.598272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:34473 (local address 127.0.0.1:45888)
0806 01:03:44.606586 (+ 8314us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:44.606599 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:44.606725 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:44.607274 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:44.607284 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:44.607327 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:44.607842 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.607856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.610119 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.610124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.611969 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.611981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.613941 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.613948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.614177 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.614233 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 01:03:44.614366 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 01:03:44.614562 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8201,"thread_start_us":5323,"threads_started":1}
I20260806 01:03:44.614979 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.598680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45888 (local address 127.5.108.131:34473)
0806 01:03:44.599402 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.599472 (+ 70us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.606710 (+ 7238us) server_negotiation.cc:407] Connection header received
0806 01:03:44.606837 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.606842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.607044 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.607220 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:44.608114 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.609942 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.612130 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.613661 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.613802 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.614077 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:44.614446 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 01:03:44.614625 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":937,"threads_started":1}
I20260806 01:03:44.619002 25236 tablet_service.cc:1467] Processing CreateTablet for tablet 0b1bcebc2e98407fbfcb1aa808516252 (DEFAULT_TABLE table=test-table [id=9218a0471d1c42bf8be27c0d55a1daa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:03:44.620297 25236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b1bcebc2e98407fbfcb1aa808516252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:44.620653 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.586873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37345 (local address 127.0.0.1:50548)
0806 01:03:44.603844 (+ 16971us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:44.603861 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:44.603977 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:44.607853 (+ 3876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:44.607856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:44.607875 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:44.608319 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.608334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.610201 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.610208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.613924 (+ 3716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.613939 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.615875 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.615881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.616022 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.616051 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 01:03:44.616110 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 01:03:44.616231 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16845,"thread_start_us":19472,"threads_started":1}
I20260806 01:03:44.622033 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.587607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50548 (local address 127.5.108.129:37345)
0806 01:03:44.605484 (+ 17877us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.605497 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.605536 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:03:44.605681 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.605691 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.607650 (+ 1959us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.607858 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:44.608535 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.610046 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.614128 (+ 4082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.615570 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.621294 (+ 5724us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.621511 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:44.621577 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 01:03:44.621728 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17663,"thread_start_us":17164,"threads_started":1}
I20260806 01:03:44.626004 24858 tablet_service.cc:1467] Processing CreateTablet for tablet 0b1bcebc2e98407fbfcb1aa808516252 (DEFAULT_TABLE table=test-table [id=9218a0471d1c42bf8be27c0d55a1daa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:03:44.627559 24858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b1bcebc2e98407fbfcb1aa808516252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:44.635962 25353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.582901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44319 (local address 127.0.0.1:56540)
0806 01:03:44.597138 (+ 14237us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:44.597181 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:44.597373 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:44.628452 (+ 31079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:44.628458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:44.628487 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:44.628827 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.628838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.630790 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.630796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.632934 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.632949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.635061 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.635068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.635241 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.635277 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:03:44.635344 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 01:03:44.635531 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13982,"thread_start_us":14664,"threads_started":1}
I20260806 01:03:44.637255 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.583465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.5.108.130:44319)
0806 01:03:44.627832 (+ 44367us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.627841 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.627870 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:03:44.627996 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.628003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.628252 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.628462 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:44.629045 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.630616 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.633166 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.634842 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.636506 (+ 1664us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.636711 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:44.636781 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 01:03:44.636954 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44191,"thread_start_us":45194,"threads_started":1}
I20260806 01:03:44.641885 25050 tablet_service.cc:1467] Processing CreateTablet for tablet 0b1bcebc2e98407fbfcb1aa808516252 (DEFAULT_TABLE table=test-table [id=9218a0471d1c42bf8be27c0d55a1daa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:03:44.643672 25050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b1bcebc2e98407fbfcb1aa808516252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:44.644593 25362 tablet_bootstrap.cc:492] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: Bootstrap starting.
I20260806 01:03:44.648600 25362 tablet_bootstrap.cc:654] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:44.650851 25362 log.cc:824] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:44.656975 25362 tablet_bootstrap.cc:492] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: No bootstrap required, opened a new log
I20260806 01:03:44.657289 25362 ts_tablet_manager.cc:1397] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260806 01:03:44.668174 25367 tablet_bootstrap.cc:492] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: Bootstrap starting.
I20260806 01:03:44.669538 25373 tablet_bootstrap.cc:492] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: Bootstrap starting.
I20260806 01:03:44.669749 25362 raft_consensus.cc:348] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } }
I20260806 01:03:44.670221 25362 raft_consensus.cc:374] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:44.670322 25362 raft_consensus.cc:729] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 677c003585d649d6928b7a74bb466b7a, State: Initialized, Role: FOLLOWER
I20260806 01:03:44.670922 25362 consensus_queue.cc:260] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } }
I20260806 01:03:44.673547 25373 tablet_bootstrap.cc:654] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:44.674849 25373 log.cc:824] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:44.678481 25367 tablet_bootstrap.cc:654] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:44.679811 25367 log.cc:824] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:44.684314 25373 tablet_bootstrap.cc:492] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: No bootstrap required, opened a new log
I20260806 01:03:44.685762 25362 ts_tablet_manager.cc:1428] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: Time spent starting tablet: real 0.028s user 0.014s sys 0.011s
I20260806 01:03:44.686208 25327 heartbeater.cc:503] Master 127.5.108.190:44197 was elected leader, sending a full tablet report...
I20260806 01:03:44.686319 25373 ts_tablet_manager.cc:1397] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.008s
I20260806 01:03:44.686767 25367 tablet_bootstrap.cc:492] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: No bootstrap required, opened a new log
I20260806 01:03:44.687042 25367 ts_tablet_manager.cc:1397] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.000s
I20260806 01:03:44.695021 25373 raft_consensus.cc:348] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } }
I20260806 01:03:44.695410 25373 raft_consensus.cc:374] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:44.695533 25373 raft_consensus.cc:729] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b34bc0f5630145288df2da46665f0cfe, State: Initialized, Role: FOLLOWER
I20260806 01:03:44.696137 25373 consensus_queue.cc:260] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } }
I20260806 01:03:44.697353 25367 raft_consensus.cc:348] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } }
I20260806 01:03:44.697767 25367 raft_consensus.cc:374] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:44.697934 25367 raft_consensus.cc:729] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f718192538c640faa96eb2cfa5d97a58, State: Initialized, Role: FOLLOWER
I20260806 01:03:44.698493 25140 heartbeater.cc:503] Master 127.5.108.190:44197 was elected leader, sending a full tablet report...
I20260806 01:03:44.698987 25373 ts_tablet_manager.cc:1428] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 01:03:44.699635 25367 consensus_queue.cc:260] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } }
I20260806 01:03:44.700668 24945 heartbeater.cc:503] Master 127.5.108.190:44197 was elected leader, sending a full tablet report...
I20260806 01:03:44.701344 25367 ts_tablet_manager.cc:1428] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260806 01:03:44.789619 25374 raft_consensus.cc:482] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:03:44.789966 25374 raft_consensus.cc:504] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } }
I20260806 01:03:44.791842 25374 leader_election.cc:288] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b34bc0f5630145288df2da46665f0cfe (127.5.108.130:44319), f718192538c640faa96eb2cfa5d97a58 (127.5.108.129:37345)
I20260806 01:03:44.799825 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.793200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:60971 (local address 127.5.108.130:44319)
0806 01:03:44.793432 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.793436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.793453 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:03:44.793526 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.793529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.793596 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.793714 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:44.794467 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.795600 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.797405 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.798699 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.798890 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.798982 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:44.799318 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 01:03:44.799504 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:03:44.799880 25381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.792187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44319 (local address 127.5.108.131:60971)
0806 01:03:44.793122 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:44.793141 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:44.793255 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:44.793730 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:44.793735 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:44.793761 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:44.794210 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.794225 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.795778 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.795782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.797226 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.797239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.798923 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.798929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.799113 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.799149 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:03:44.799260 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 01:03:44.799415 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":1,"thread_start_us":651,"threads_started":1}
I20260806 01:03:44.799943 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.792151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37345 (local address 127.5.108.131:43349)
0806 01:03:44.792379 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:44.792403 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:44.792584 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:44.793360 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:44.793365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:44.793392 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:44.793580 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.793590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.795222 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.795229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.797323 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:44.797339 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.799132 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.799137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:44.799325 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.799361 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:03:44.799483 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 01:03:44.799642 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":49}
I20260806 01:03:44.800485 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.792693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:43349 (local address 127.5.108.129:37345)
0806 01:03:44.792919 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.792925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.792944 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:03:44.793018 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.793023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.793094 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.793188 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:44.793790 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.795025 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.797490 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.798807 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.798964 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.799054 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:44.800112 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0806 01:03:44.800249 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:03:44.801447 25073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b1bcebc2e98407fbfcb1aa808516252" candidate_uuid: "677c003585d649d6928b7a74bb466b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b34bc0f5630145288df2da46665f0cfe" is_pre_election: true
I20260806 01:03:44.801468 24887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b1bcebc2e98407fbfcb1aa808516252" candidate_uuid: "677c003585d649d6928b7a74bb466b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f718192538c640faa96eb2cfa5d97a58" is_pre_election: true
I20260806 01:03:44.802129 24887 raft_consensus.cc:2454] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677c003585d649d6928b7a74bb466b7a in term 0.
I20260806 01:03:44.802126 25073 raft_consensus.cc:2454] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677c003585d649d6928b7a74bb466b7a in term 0.
I20260806 01:03:44.802928 25172 leader_election.cc:302] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 677c003585d649d6928b7a74bb466b7a, f718192538c640faa96eb2cfa5d97a58; no voters:
I20260806 01:03:44.803413 25374 raft_consensus.cc:2781] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:03:44.803572 25374 raft_consensus.cc:482] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:03:44.803681 25374 raft_consensus.cc:3037] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:44.806813 25374 raft_consensus.cc:504] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } }
I20260806 01:03:44.807548 25374 leader_election.cc:288] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [CANDIDATE]: Term 1 election: Requested vote from peers b34bc0f5630145288df2da46665f0cfe (127.5.108.130:44319), f718192538c640faa96eb2cfa5d97a58 (127.5.108.129:37345)
I20260806 01:03:44.808076 25073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b1bcebc2e98407fbfcb1aa808516252" candidate_uuid: "677c003585d649d6928b7a74bb466b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b34bc0f5630145288df2da46665f0cfe"
I20260806 01:03:44.808315 24887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b1bcebc2e98407fbfcb1aa808516252" candidate_uuid: "677c003585d649d6928b7a74bb466b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f718192538c640faa96eb2cfa5d97a58"
I20260806 01:03:44.808346 25073 raft_consensus.cc:3037] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:44.808523 24887 raft_consensus.cc:3037] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:44.810933 25073 raft_consensus.cc:2454] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677c003585d649d6928b7a74bb466b7a in term 1.
I20260806 01:03:44.811436 25175 leader_election.cc:302] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 677c003585d649d6928b7a74bb466b7a, b34bc0f5630145288df2da46665f0cfe; no voters:
I20260806 01:03:44.811645 24887 raft_consensus.cc:2454] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677c003585d649d6928b7a74bb466b7a in term 1.
I20260806 01:03:44.811930 25374 raft_consensus.cc:2781] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:03:44.812561 25374 raft_consensus.cc:686] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 1 LEADER]: Becoming Leader. State: Replica: 677c003585d649d6928b7a74bb466b7a, State: Running, Role: LEADER
I20260806 01:03:44.813323 25374 consensus_queue.cc:237] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } }
I20260806 01:03:44.818737 24677 catalog_manager.cc:5495] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 677c003585d649d6928b7a74bb466b7a (127.5.108.131). New cstate: current_term: 1 leader_uuid: "677c003585d649d6928b7a74bb466b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "677c003585d649d6928b7a74bb466b7a" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 } health_report { overall_health: UNKNOWN } } }
W20260806 01:03:44.856766 25333 tablet.cc:2369] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:03:44.893880 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.887956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.5.108.131:34473)
0806 01:03:44.888163 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.888169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.888206 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:03:44.888449 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.888454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.888555 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.888723 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:03:44.889066 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.890089 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.891518 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.892094 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.892216 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.892672 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:03:44.893325 (+ 653us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:03:44.893456 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:44.893550 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 01:03:44.893627 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:03:44.903510 25073 raft_consensus.cc:1261] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [term 1 FOLLOWER]: Refusing update from remote peer 677c003585d649d6928b7a74bb466b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:03:44.903510 24887 raft_consensus.cc:1261] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [term 1 FOLLOWER]: Refusing update from remote peer 677c003585d649d6928b7a74bb466b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:03:44.904667 25383 consensus_queue.cc:1035] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b34bc0f5630145288df2da46665f0cfe" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:03:44.905115 25374 consensus_queue.cc:1035] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f718192538c640faa96eb2cfa5d97a58" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:03:44.922854 25143 tablet.cc:2369] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:03:44.924245 25386 mvcc.cc:205] Tried to move back new op lower bound from 7315366809194090496 to 7315366808839868416. Current Snapshot: MvccSnapshot[applied={T|T < 7315366809194090496}]
I20260806 01:03:44.956712 24677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:03:44.957021 24677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:03:44.960520 24677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0718165bf7174f3181afdcbce8e596db: Sending DeleteTablet for 3 replicas of tablet 0b1bcebc2e98407fbfcb1aa808516252
I20260806 01:03:44.961759 25050 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b1bcebc2e98407fbfcb1aa808516252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:03:44 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:56540
I20260806 01:03:44.961755 24858 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b1bcebc2e98407fbfcb1aa808516252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:03:44 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:50548
I20260806 01:03:44.962968 25236 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b1bcebc2e98407fbfcb1aa808516252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:03:44 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:45888
I20260806 01:03:44.967444 25401 tablet_replica.cc:330] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: stopping tablet replica
I20260806 01:03:44.967869 25401 raft_consensus.cc:2229] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:03:44.968412 25401 raft_consensus.cc:2258] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:03:44.973692 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 24729
I20260806 01:03:44.974116 25398 tablet_replica.cc:330] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: stopping tablet replica
I20260806 01:03:44.974570 25398 raft_consensus.cc:2229] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:03:44.975075 25398 raft_consensus.cc:2258] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:03:44.976048 25401 ts_tablet_manager.cc:1905] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:03:44.977886 25403 tablet_replica.cc:330] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: stopping tablet replica
I20260806 01:03:44.981518 25398 ts_tablet_manager.cc:1905] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:03:44.982865 25403 raft_consensus.cc:2229] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:03:44.983573 25403 raft_consensus.cc:2258] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:03:44.985677 25403 ts_tablet_manager.cc:1905] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:03:44.985863 25401 ts_tablet_manager.cc:1918] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:03:44.986127 25401 log.cc:1196] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/0b1bcebc2e98407fbfcb1aa808516252
I20260806 01:03:44.986702 25401 ts_tablet_manager.cc:1939] T 0b1bcebc2e98407fbfcb1aa808516252 P b34bc0f5630145288df2da46665f0cfe: Deleting consensus metadata
I20260806 01:03:44.988591 24656 catalog_manager.cc:4841] TS b34bc0f5630145288df2da46665f0cfe (127.5.108.130:44319): tablet 0b1bcebc2e98407fbfcb1aa808516252 (table test-table [id=9218a0471d1c42bf8be27c0d55a1daa3]) successfully deleted
I20260806 01:03:44.988821 25398 ts_tablet_manager.cc:1918] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:03:44.988993 25398 log.cc:1196] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/0b1bcebc2e98407fbfcb1aa808516252
I20260806 01:03:44.989476 25398 ts_tablet_manager.cc:1939] T 0b1bcebc2e98407fbfcb1aa808516252 P f718192538c640faa96eb2cfa5d97a58: Deleting consensus metadata
I20260806 01:03:44.991341 24652 catalog_manager.cc:4841] TS f718192538c640faa96eb2cfa5d97a58 (127.5.108.129:37345): tablet 0b1bcebc2e98407fbfcb1aa808516252 (table test-table [id=9218a0471d1c42bf8be27c0d55a1daa3]) successfully deleted
I20260806 01:03:44.993598 25403 ts_tablet_manager.cc:1918] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:03:44.993790 25403 log.cc:1196] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/0b1bcebc2e98407fbfcb1aa808516252
I20260806 01:03:44.994254 25403 ts_tablet_manager.cc:1939] T 0b1bcebc2e98407fbfcb1aa808516252 P 677c003585d649d6928b7a74bb466b7a: Deleting consensus metadata
I20260806 01:03:44.995510 24656 catalog_manager.cc:4841] TS 677c003585d649d6928b7a74bb466b7a (127.5.108.131:34473): tablet 0b1bcebc2e98407fbfcb1aa808516252 (table test-table [id=9218a0471d1c42bf8be27c0d55a1daa3]) successfully deleted
I20260806 01:03:45.001626 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:44.975082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50558 (local address 127.5.108.129:37345)
0806 01:03:44.975262 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:44.975267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:44.979932 (+ 4665us) server_negotiation.cc:407] Connection header received
0806 01:03:44.980230 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:44.980235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:44.980327 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:44.980403 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:44.988666 (+ 8263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.989874 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.991821 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:44.992468 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:44.992597 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:44.996026 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:44.996035 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:44.996048 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:44.996081 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:44.998815 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:44.999691 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:44.999698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:44.999700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:44.999836 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:45.000230 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:45.000233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:45.000235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:45.000588 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:45.000820 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:45.001160 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 01:03:45.001321 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:03:45.002271 24939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:45.091094 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 24729
I20260806 01:03:45.121050 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 24950
I20260806 01:03:45.137146 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:45.121964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56550 (local address 127.5.108.130:44319)
0806 01:03:45.122147 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:45.122151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:45.122175 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:03:45.122316 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:45.122320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:45.122383 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:45.122467 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:45.124349 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:45.125104 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:45.126956 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:45.127342 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:45.127605 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:45.130827 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:45.130840 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:45.130854 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:45.130889 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:45.133930 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:45.134844 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:45.134854 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:45.134860 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:45.134992 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:45.135497 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:45.135501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:45.135503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:45.135883 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:45.136118 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:45.136457 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 01:03:45.136737 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:03:45.137923 25130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:45.251014 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 24950
I20260806 01:03:45.275120 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 25147
I20260806 01:03:45.291157 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:45.276032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.5.108.131:34473)
0806 01:03:45.276248 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:45.276254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:45.276278 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:03:45.276427 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:45.276433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:45.276521 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:45.276641 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:45.278091 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:45.279337 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:45.280886 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:45.281569 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:45.281720 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:45.284464 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:45.284484 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:45.284505 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:45.284557 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:45.288343 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:45.289033 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:45.289041 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:45.289043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:45.289203 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:45.289594 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:45.289600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:45.289603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:45.289972 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:45.290306 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:45.290606 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 01:03:45.290832 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 01:03:45.292047 25319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:45.410640 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 25147
I20260806 01:03:45.438812 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 24629
I20260806 01:03:45.440049 24721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:45.551507 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 24629
I20260806 01:03:45.585155 5554 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 01:03:45.931448 5554 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 01:03:45.931655 5554 mini_ranger.cc:66] Stopping Ranger...
I20260806 01:03:46.276942 5554 mini_ranger.cc:68] Stopped Ranger
2026-08-06 01:03:46.277 UTC [23298] LOG: received smart shutdown request
2026-08-06 01:03:46.280 UTC [23298] LOG: background worker "logical replication launcher" (PID 23310) exited with exit code 1
2026-08-06 01:03:46.280 UTC [23303] LOG: shutting down
2026-08-06 01:03:46.426 UTC [23298] LOG: database system is shut down
2026-08-06T01:03:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51521 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:03:46 dist-test-slave-x4qh krb5kdc[25422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:03:46 dist-test-slave-x4qh krb5kdc[25422](info): set up 2 sockets
Aug 06 01:03:46 dist-test-slave-x4qh krb5kdc[25422](info): commencing operation
krb5kdc: starting...
W20260806 01:03:47.844005 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:03:48 dist-test-slave-x4qh krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:03:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:03:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:03:48.090672 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37627
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:37627,127.5.108.189:33339,127.5.108.188:35013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:03:48.416049 25442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:48.416278 25442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:48.416412 25442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:48.423108 25442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:03:48.423185 25442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:48.423225 25442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:48.423267 25442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:03:48.423381 25442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:03:48.432173 25442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:37627,127.5.108.189:33339,127.5.108.188:35013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:48.433246 25442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:48.434316 25442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:48.445873 25448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:48.445859 25449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:48.446319 25452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:48.446798 25442 server_base.cc:1034] running on GCE node
I20260806 01:03:48.447587 25442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:48.448863 25442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:48.450085 25442 hybrid_clock.cc:648] HybridClock initialized: now 1785978228450036 us; error 43 us; skew 500 ppm
I20260806 01:03:48.450413 25442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:48 dist-test-slave-x4qh krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:48.456053 25442 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:03:48.457394 25442 webserver.cc:460] Webserver started at http://127.5.108.190:36431/ using document root <none> and password file <none>
I20260806 01:03:48.458030 25442 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:48.458189 25442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:48.458505 25442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:03:48.461378 25442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "730e2666103e480c9f13373b0109b9ba"
format_stamp: "Formatted at 2026-08-06 01:03:48 on dist-test-slave-x4qh"
server_key: "e3b58e9842c4f4a8ce0b315c85f5d7f0"
server_key_iv: "914a65237b1fa51e481e18f1d74dd75f"
server_key_version: "encryptionkey@0"
I20260806 01:03:48.462122 25442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "730e2666103e480c9f13373b0109b9ba"
format_stamp: "Formatted at 2026-08-06 01:03:48 on dist-test-slave-x4qh"
server_key: "e3b58e9842c4f4a8ce0b315c85f5d7f0"
server_key_iv: "914a65237b1fa51e481e18f1d74dd75f"
server_key_version: "encryptionkey@0"
I20260806 01:03:48.466754 25442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 01:03:48.470114 25464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:48.471472 25442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:03:48.471657 25442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "730e2666103e480c9f13373b0109b9ba"
format_stamp: "Formatted at 2026-08-06 01:03:48 on dist-test-slave-x4qh"
server_key: "e3b58e9842c4f4a8ce0b315c85f5d7f0"
server_key_iv: "914a65237b1fa51e481e18f1d74dd75f"
server_key_version: "encryptionkey@0"
I20260806 01:03:48.471854 25442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:48.485903 25442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:48.490091 25442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:48.490358 25442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:48.508653 25442 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:37627
I20260806 01:03:48.508698 25531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:37627 every 8 connection(s)
I20260806 01:03:48.509919 25442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:03:48.511793 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 25442
I20260806 01:03:48.512036 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:03:48.512720 5554 external_mini_cluster.cc:1428] Setting key c99fa4b268eede82e4211b76afdffdda
I20260806 01:03:48.513521 25532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:48.519743 25532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:03:48 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.108.189@KRBTEST.COM; defaulting to no policy
I20260806 01:03:48.541742 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.522470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37627 (local address 127.0.0.1:43780)
0806 01:03:48.523279 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.523322 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:48.525008 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:48.525827 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:48.525836 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:48.526628 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:48.527071 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:48.527095 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:48.528676 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:48.528684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:48.530546 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:48.530563 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:48.531421 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:48.531431 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:48.531628 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:48.532226 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:48.532264 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:48.534688 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:48.537742 (+ 3054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:48.537754 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:48.537765 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:48.538138 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:48.538537 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:48.538543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:48.538547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:48.538702 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:48.539419 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:48.539427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:48.539655 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 01:03:48.539887 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 01:03:48.540363 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":23,"thread_start_us":347,"threads_started":1}
I20260806 01:03:48.541761 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.522474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.5.108.190:37627)
0806 01:03:48.523896 (+ 1422us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:48.523902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:48.523946 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:03:48.525355 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:48.525361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:48.525573 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:48.525752 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:48.527272 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:48.528505 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:48.530691 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:48.531167 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:48.531285 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:48.534893 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:48.534901 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:48.534911 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:48.534936 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:48.537614 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:48.538266 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:48.538273 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:48.538277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:48.538427 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:48.538808 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:48.538816 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:48.538819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:48.539258 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:48.539502 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:48.539992 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 01:03:48.540455 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":201,"server-negotiator.queue_time_us":1256,"spinlock_wait_cycles":14080,"thread_start_us":1052,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.189@KRBTEST.COM" created.
I20260806 01:03:48.544677 25532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:03:48.545621 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.545216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:46954)
0806 01:03:48.545380 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.545504 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:03:48.545835 25468 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
I20260806 01:03:48.547341 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.547015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:46962)
0806 01:03:48.547159 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.547229 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 01:03:48.547576 25532 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:33339: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab.
Entry for principal kudu/127.5.108.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab.
I20260806 01:03:48.572904 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.189:33339
--webserver_interface=127.5.108.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab
--principal=kudu/127.5.108.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:37627,127.5.108.189:33339,127.5.108.188:35013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:03:48.596901 25532 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } attempt: 1
I20260806 01:03:48.598063 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.597542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:46976)
0806 01:03:48.597756 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.597901 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 01:03:48.599006 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.598664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:46982)
0806 01:03:48.598803 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.598893 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 01:03:48.599211 25532 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:33339: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
I20260806 01:03:48.667568 25532 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } attempt: 2
I20260806 01:03:48.668833 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.668330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:46996)
0806 01:03:48.668581 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.668684 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:03:48.669938 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.669530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:47000)
0806 01:03:48.669718 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.669786 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:03:48.670140 25532 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:33339: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
I20260806 01:03:48.761413 25532 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } attempt: 3
I20260806 01:03:48.762570 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.762120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:47014)
0806 01:03:48.762321 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.762426 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 01:03:48.763634 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.763272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:47030)
0806 01:03:48.763450 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.763512 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:03:48.763828 25532 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:33339: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
W20260806 01:03:48.894622 25545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:48.894860 25545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:48.894999 25545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:48.901824 25545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:03:48.901932 25545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:48.902005 25545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:48.902101 25545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:03:48.902184 25545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:03:48.907081 25532 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } attempt: 4
I20260806 01:03:48.908434 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.907833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:47042)
0806 01:03:48.908163 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.908266 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260806 01:03:48.909507 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:48.909101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:47054)
0806 01:03:48.909256 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:48.909363 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:03:48.909711 25532 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:33339: Network error: Client connection negotiation failed: client connection to 127.5.108.189:33339: connect: Connection refused (error 111)
I20260806 01:03:48.912178 25545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:37627,127.5.108.189:33339,127.5.108.188:35013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab
--principal=kudu/127.5.108.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.189:33339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.108.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:48.913304 25545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:48.914601 25545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:48.926100 25558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:48.926174 25561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:48.926177 25559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:48.926927 25545 server_base.cc:1034] running on GCE node
I20260806 01:03:48.927695 25545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:48.928898 25545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:48.930091 25545 hybrid_clock.cc:648] HybridClock initialized: now 1785978228930055 us; error 34 us; skew 500 ppm
I20260806 01:03:48.930434 25545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:48 dist-test-slave-x4qh krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:48.933517 25545 init.cc:377] Logged in from keytab as kudu/127.5.108.189@KRBTEST.COM (short username kudu)
I20260806 01:03:48.934959 25545 webserver.cc:460] Webserver started at http://127.5.108.189:39121/ using document root <none> and password file <none>
I20260806 01:03:48.935701 25545 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:48.935849 25545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:48.936165 25545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:03:48.939014 25545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/data/instance:
uuid: "50b8f01206ce404aa56402376dcb0795"
format_stamp: "Formatted at 2026-08-06 01:03:48 on dist-test-slave-x4qh"
server_key: "e09499e0176896b9cd20c591a7cfa5c5"
server_key_iv: "29a068445220c032102c24e49c89a6d7"
server_key_version: "encryptionkey@0"
I20260806 01:03:48.939851 25545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/wal/instance:
uuid: "50b8f01206ce404aa56402376dcb0795"
format_stamp: "Formatted at 2026-08-06 01:03:48 on dist-test-slave-x4qh"
server_key: "e09499e0176896b9cd20c591a7cfa5c5"
server_key_iv: "29a068445220c032102c24e49c89a6d7"
server_key_version: "encryptionkey@0"
I20260806 01:03:48.945158 25545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260806 01:03:48.948406 25571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:48.949651 25545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:03:48.949841 25545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/wal
uuid: "50b8f01206ce404aa56402376dcb0795"
format_stamp: "Formatted at 2026-08-06 01:03:48 on dist-test-slave-x4qh"
server_key: "e09499e0176896b9cd20c591a7cfa5c5"
server_key_iv: "29a068445220c032102c24e49c89a6d7"
server_key_version: "encryptionkey@0"
I20260806 01:03:48.950035 25545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:48.968823 25545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:48.972688 25545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:48.972970 25545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:48.991895 25545 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.189:33339
I20260806 01:03:48.991946 25631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.189:33339 every 8 connection(s)
I20260806 01:03:48.993211 25545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
I20260806 01:03:48.994983 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 25545
I20260806 01:03:48.995229 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-1/wal/instance
I20260806 01:03:48.995884 5554 external_mini_cluster.cc:1428] Setting key cabeb3ca3d42bc93e70aefbb8de58fef
I20260806 01:03:48.997668 25632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:49.007439 25632 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.108.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.188@KRBTEST.COM" created.
I20260806 01:03:49.028386 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.010351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43790 (local address 127.5.108.190:37627)
0806 01:03:49.010594 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.010601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.011748 (+ 1147us) server_negotiation.cc:407] Connection header received
0806 01:03:49.013543 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.013551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.013642 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.013774 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.015174 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.016311 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.017803 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.018568 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.018714 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.022198 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.022206 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.022212 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.022252 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.025492 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.026172 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.026176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.026178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.026284 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.026677 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.026680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.026682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.027003 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.027150 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.028002 (+ 852us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.028158 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:03:49.029964 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.010777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37627 (local address 127.0.0.1:43790)
0806 01:03:49.011552 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.011603 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:49.013283 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:49.013952 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:49.013959 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:49.014431 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:49.014993 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.015010 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.016466 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.016470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.017625 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.017658 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.018785 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.018790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.018974 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.019737 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:49.019757 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:49.021983 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:49.025625 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.025639 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.025649 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.026077 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.026373 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.026375 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.026378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.026582 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.027138 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:49.027147 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:49.027588 (+ 441us) client_negotiation.cc:769] Sending connection context
0806 01:03:49.027922 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 01:03:49.028437 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":340,"threads_started":1}
I20260806 01:03:49.031769 25632 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
I20260806 01:03:49.045127 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.032381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:47064)
0806 01:03:49.032542 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.032567 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:49.032637 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:49.033613 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:49.033617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:49.034004 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:49.034121 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.034128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.035388 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.035396 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.036198 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.036209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.037043 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.037048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.037297 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.037852 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:49.037869 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:49.039896 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:49.042752 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.042758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.042764 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.043075 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.043470 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.043474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.043478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.043612 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.044190 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:49.044195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:49.044330 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 01:03:49.044486 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 01:03:49.044743 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":41}
I20260806 01:03:49.045243 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.032673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.5.108.189:33339)
0806 01:03:49.033271 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.033275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.033298 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:03:49.033334 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.033338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.033479 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.033619 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.034276 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.035225 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.036349 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.036891 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.037036 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.040079 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.040089 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.040100 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.040128 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.042595 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.043198 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.043211 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.043213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.043327 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.043723 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.043730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.043742 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.044046 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.044263 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.044618 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.044858 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":461,"thread_start_us":339,"threads_started":1}
I20260806 01:03:49.047163 25632 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:03:49.048391 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.047964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40228)
0806 01:03:49.048146 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.048254 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 01:03:49.048818 25576 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
I20260806 01:03:49.050402 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.050114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40244)
0806 01:03:49.050246 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.050314 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 01:03:49.050741 25632 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:35013: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab.
Entry for principal kudu/127.5.108.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab.
I20260806 01:03:49.059890 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.188:35013
--webserver_interface=127.5.108.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab
--principal=kudu/127.5.108.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:37627,127.5.108.189:33339,127.5.108.188:35013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:03:49.100047 25632 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } attempt: 1
I20260806 01:03:49.101190 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.100773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40254)
0806 01:03:49.100951 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.101049 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:03:49.102177 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.101899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40270)
0806 01:03:49.102027 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.102084 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 01:03:49.102437 25632 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:35013: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
I20260806 01:03:49.170768 25632 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } attempt: 2
I20260806 01:03:49.172156 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.171579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40286)
0806 01:03:49.171832 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.171947 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 01:03:49.173367 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.172932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40296)
0806 01:03:49.173107 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.173204 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 01:03:49.173676 25632 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:35013: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
I20260806 01:03:49.208952 25532 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } attempt: 5
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
I20260806 01:03:49.222079 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.209850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47070 (local address 127.5.108.189:33339)
0806 01:03:49.210018 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.210024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.210070 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:03:49.210323 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.210327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.210393 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.210514 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.211335 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.212077 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.213200 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.213658 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.213816 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.216854 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.216858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.216862 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.216894 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.219611 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.220243 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.220250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.220256 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.220379 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.220747 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.220752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.220755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.221021 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.221177 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.221567 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.221781 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:03:49.222075 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.209758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:47070)
0806 01:03:49.209986 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.210006 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:49.210150 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:49.210555 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:49.210561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:49.210967 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:49.211170 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.211179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.212215 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.212221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.213049 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.213060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.213837 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.213843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.214032 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.214657 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:49.214676 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:49.216662 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:49.219752 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.219759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.219766 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.220128 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.220491 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.220495 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.220499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.220631 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.221142 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:49.221147 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:49.221312 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 01:03:49.221457 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 01:03:49.221720 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 01:03:49.223524 25532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:03:49.224680 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.224315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40308)
0806 01:03:49.224484 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.224553 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 01:03:49.225636 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.225329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40318)
0806 01:03:49.225476 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.225544 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 01:03:49.225876 25532 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:35013: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
I20260806 01:03:49.264926 25632 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } attempt: 3
I20260806 01:03:49.266224 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.265605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40322)
0806 01:03:49.265853 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.265994 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 01:03:49.267437 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.266860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40324)
0806 01:03:49.267125 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.267244 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260806 01:03:49.267709 25632 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:35013: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
I20260806 01:03:49.277076 25532 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } attempt: 1
I20260806 01:03:49.278070 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.277664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40330)
0806 01:03:49.277834 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.277930 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 01:03:49.279069 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.278742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40346)
0806 01:03:49.278901 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.278967 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:03:49.279287 25532 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:35013: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
I20260806 01:03:49.347525 25532 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } attempt: 2
I20260806 01:03:49.348680 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.348185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40350)
0806 01:03:49.348428 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.348525 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 01:03:49.349701 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.349399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40366)
0806 01:03:49.349548 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.349608 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:03:49.349915 25532 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:35013: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
W20260806 01:03:49.394549 25644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:49.394889 25644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:49.395058 25644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:49.402166 25644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:03:49.402309 25644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:49.402417 25644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:49.402518 25644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:03:49.402588 25644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:03:49.410943 25632 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } attempt: 4
I20260806 01:03:49.412086 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.411612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40370)
0806 01:03:49.411831 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.411913 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 01:03:49.412586 25644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:37627,127.5.108.189:33339,127.5.108.188:35013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab
--principal=kudu/127.5.108.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.188:35013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.108.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:49.413084 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.412741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40376)
0806 01:03:49.412915 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.412992 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:03:49.413401 25632 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:35013: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
I20260806 01:03:49.414052 25644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:49.415320 25644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:49.426879 25654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:49.426905 25653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:49.426975 25657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:49.427798 25644 server_base.cc:1034] running on GCE node
I20260806 01:03:49.428609 25644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:49.429870 25644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:49.431066 25644 hybrid_clock.cc:648] HybridClock initialized: now 1785978229431015 us; error 42 us; skew 500 ppm
I20260806 01:03:49.431461 25644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:49.434497 25644 init.cc:377] Logged in from keytab as kudu/127.5.108.188@KRBTEST.COM (short username kudu)
I20260806 01:03:49.435860 25644 webserver.cc:460] Webserver started at http://127.5.108.188:36713/ using document root <none> and password file <none>
I20260806 01:03:49.436447 25644 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:49.436553 25644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:49.436774 25644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:03:49.439508 25644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/data/instance:
uuid: "c38b5009e1494345826621e67e7469c8"
format_stamp: "Formatted at 2026-08-06 01:03:49 on dist-test-slave-x4qh"
server_key: "50f11105921d74a6cc3be9e6264d6ac1"
server_key_iv: "91d4295ad8f160728446bc364afd6019"
server_key_version: "encryptionkey@0"
I20260806 01:03:49.440260 25644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/wal/instance:
uuid: "c38b5009e1494345826621e67e7469c8"
format_stamp: "Formatted at 2026-08-06 01:03:49 on dist-test-slave-x4qh"
server_key: "50f11105921d74a6cc3be9e6264d6ac1"
server_key_iv: "91d4295ad8f160728446bc364afd6019"
server_key_version: "encryptionkey@0"
I20260806 01:03:49.444932 25644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:03:49.448045 25670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:49.449329 25644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:03:49.449506 25644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/wal
uuid: "c38b5009e1494345826621e67e7469c8"
format_stamp: "Formatted at 2026-08-06 01:03:49 on dist-test-slave-x4qh"
server_key: "50f11105921d74a6cc3be9e6264d6ac1"
server_key_iv: "91d4295ad8f160728446bc364afd6019"
server_key_version: "encryptionkey@0"
I20260806 01:03:49.449700 25644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:49.456102 25532 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } attempt: 3
I20260806 01:03:49.457141 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.456661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40390)
0806 01:03:49.456887 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.456977 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 01:03:49.458052 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.457727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40392)
0806 01:03:49.457887 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.457957 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260806 01:03:49.458269 25532 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:35013: Network error: Client connection negotiation failed: client connection to 127.5.108.188:35013: connect: Connection refused (error 111)
I20260806 01:03:49.466706 25644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:49.470539 25644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:49.470832 25644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:49.487780 25644 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.188:35013
I20260806 01:03:49.487857 25742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.188:35013 every 8 connection(s)
I20260806 01:03:49.489079 25644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
I20260806 01:03:49.492470 25743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:49.495966 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 25644
I20260806 01:03:49.496170 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978022745104-5554-0/minicluster-data/master-2/wal/instance
I20260806 01:03:49.496703 5554 external_mini_cluster.cc:1428] Setting key 7adb3b2fb8375e8ce611c3cc0c6740eb
I20260806 01:03:49.499482 25743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
I20260806 01:03:49.521346 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.503328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.5.108.190:37627)
0806 01:03:49.503611 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.503617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.504105 (+ 488us) server_negotiation.cc:407] Connection header received
0806 01:03:49.504366 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.504374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.504444 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.504554 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.505834 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.506892 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.508654 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.509358 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.509504 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.514284 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.514291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.514295 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.514332 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.518027 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.518882 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.518888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.518891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.519105 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.519644 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.519649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.519653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.519979 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.520157 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.520772 (+ 615us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.520998 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:03:49.522269 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.503221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37627 (local address 127.0.0.1:43792)
0806 01:03:49.503968 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.504010 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:49.504194 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:49.504619 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:49.504625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:49.505267 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:49.505641 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.505658 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.507101 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.507106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.508491 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.508504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.509815 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.509822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.510079 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.510918 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:49.510950 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:49.514069 (+ 3119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:49.518165 (+ 4096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.518178 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.518189 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.518745 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.519219 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.519224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.519228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.519490 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.520128 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:49.520136 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:49.520358 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 01:03:49.520626 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 01:03:49.521081 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":80,"thread_start_us":352,"threads_started":1}
I20260806 01:03:49.522276 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.498397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40396 (local address 127.5.108.188:35013)
0806 01:03:49.499202 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.499219 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.499280 (+ 61us) server_negotiation.cc:407] Connection header received
0806 01:03:49.501468 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.501499 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.501943 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.502321 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.503900 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.505388 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.507472 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.509375 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.510283 (+ 908us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.514887 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.514896 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.514907 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.514936 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.517875 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.518748 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.518756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.518760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.518947 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.519464 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.519469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.519472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.520178 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.520435 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.521415 (+ 980us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.521646 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":580,"thread_start_us":341,"threads_started":1}
I20260806 01:03:49.524572 25743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:49.537117 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.525269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:33339 (local address 127.0.0.1:47086)
0806 01:03:49.525494 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.525512 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:49.525603 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:49.525923 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:49.525927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:49.526289 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:49.526425 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.526433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.527360 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.527385 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.528218 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.528229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.528967 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.528973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.529164 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.529709 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:49.529725 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:49.531975 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:49.534776 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.534782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.534785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.535150 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.535606 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.535610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.535613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.535761 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.536244 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:49.536250 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:49.536394 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 01:03:49.536534 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 01:03:49.536792 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260806 01:03:49.537204 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.525384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.5.108.189:33339)
0806 01:03:49.525566 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.525572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.525591 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:03:49.525716 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.525721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.525791 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.525890 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.526578 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.527225 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.528370 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.528815 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.528948 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.532140 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.532164 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.532174 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.532204 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.534648 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.535271 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.535278 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.535281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.535478 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.535869 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.535874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.535877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.536118 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.536279 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.536633 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.536836 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 01:03:49.538084 25743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:03:49.539882 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.525333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43804 (local address 127.5.108.190:37627)
0806 01:03:49.525561 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.525568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.525636 (+ 68us) server_negotiation.cc:407] Connection header received
0806 01:03:49.525817 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.525822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.525885 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.525980 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.527107 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.527896 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.529150 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.529554 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.529701 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.534220 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.534225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.534228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.534255 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.536756 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.537805 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.537813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.537817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.537943 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.538378 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.538383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.538386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.538757 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.539052 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.539362 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.539606 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
I20260806 01:03:49.552663 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.538894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40402)
0806 01:03:49.539089 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.539101 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:49.539213 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:49.539563 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:49.539567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:49.540016 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:49.540169 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.540179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.541241 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.541246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.542598 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.542627 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.543267 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.543275 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.543485 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.543991 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:49.544005 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:49.546383 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:49.550008 (+ 3625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.550012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.550014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.550520 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.550938 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.550942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.550946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.551074 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.551671 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:49.551675 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:49.551823 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 01:03:49.551964 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 01:03:49.552243 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 01:03:49.552680 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.538847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40402 (local address 127.5.108.188:35013)
0806 01:03:49.539009 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.539016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.539166 (+ 150us) server_negotiation.cc:407] Connection header received
0806 01:03:49.539336 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.539340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.539447 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.539547 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.540301 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.541062 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.542766 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.543127 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.543212 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.546559 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.546565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.546571 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.546606 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.549879 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.550653 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.550658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.550661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.550808 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.551193 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.551198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.551201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.551553 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.551698 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.552041 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.552385 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":65}
I20260806 01:03:49.557196 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.541520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.5.108.189:33339)
0806 01:03:49.541684 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.541687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.541701 (+ 14us) server_negotiation.cc:407] Connection header received
0806 01:03:49.541808 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.541811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.541857 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.541924 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.543333 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.544000 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.545670 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.546266 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.546377 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.551576 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.551580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.551583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.551611 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.554821 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.555432 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.555437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.555440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.555568 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.555907 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.555912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.555915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.556148 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.556405 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.556687 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.556898 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:03:49.557480 25743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8: Bootstrap starting.
I20260806 01:03:49.561237 25743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:49.561854 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:03:49.562367 25743 log.cc:824] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:49.565043 25743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8: No bootstrap required, opened a new log
I20260806 01:03:49.575731 25743 raft_consensus.cc:348] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } }
I20260806 01:03:49.576502 25743 raft_consensus.cc:374] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:49.576764 25743 raft_consensus.cc:729] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38b5009e1494345826621e67e7469c8, State: Initialized, Role: FOLLOWER
I20260806 01:03:49.577672 25743 consensus_queue.cc:260] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } }
I20260806 01:03:49.582219 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.582685 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:03:49.583649 25743 sys_catalog.cc:564] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:03:49.589426 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.566981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43818 (local address 127.5.108.190:37627)
0806 01:03:49.567149 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.567155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.567774 (+ 619us) server_negotiation.cc:407] Connection header received
0806 01:03:49.568063 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.568070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.568147 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.568212 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.570695 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.571791 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.573990 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.574674 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.574766 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.581677 (+ 6911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.581684 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.581689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.581758 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.585437 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.586457 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.586464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.586469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.586629 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.587148 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.587154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.587157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.587553 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.587790 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.588803 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.589040 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:03:49.594748 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.568188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47106 (local address 127.5.108.189:33339)
0806 01:03:49.572021 (+ 3833us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.572029 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.577341 (+ 5312us) server_negotiation.cc:407] Connection header received
0806 01:03:49.577400 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.577404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.577487 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.577589 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.579906 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.580995 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.583059 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.583812 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.583941 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.587657 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.587663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.587667 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.587711 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.590886 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.591789 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.591794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.591798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.591947 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.592412 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.592418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.592422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.592816 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.593050 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.593897 (+ 847us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.594203 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3694}
I20260806 01:03:49.598126 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.567262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40404 (local address 127.5.108.188:35013)
0806 01:03:49.567450 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.567458 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.576940 (+ 9482us) server_negotiation.cc:407] Connection header received
0806 01:03:49.577414 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.577420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.577536 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.577642 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.579957 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.581021 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.584458 (+ 3437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.585273 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.585391 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.591362 (+ 5971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.591388 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.591393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.591432 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.594789 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.595736 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.595740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.595744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.595929 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.596403 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.596410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.596413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.596795 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.597082 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.597470 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.597688 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":45}
W20260806 01:03:49.604956 25772 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:03:49.605073 25772 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:03:49.599630 25644 master_runner.cc:186] Error getting master registration for 127.5.108.190:37627: OK, instance_id { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" instance_seqno: 1785978228503354 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:03:49.609663 25644 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:03:49.635462 25532 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } attempt: 4
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
I20260806 01:03:49.649427 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.636219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.5.108.188:35013)
0806 01:03:49.636403 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.636410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.636427 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:03:49.636476 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.636481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.636567 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.636677 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.637429 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.638426 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.640175 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.640626 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.640798 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.644024 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.644028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.644031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.644061 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.646750 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.647336 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.647342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.647346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.647566 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.647972 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.647977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.647981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.648252 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.648422 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.648799 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.649061 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:03:49.649857 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.636038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40406)
0806 01:03:49.636221 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.636236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:49.636357 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:49.636721 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:49.636727 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:49.637136 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:49.637289 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.637298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.638874 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.638879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.640031 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.640043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.640809 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.640815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.641009 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.641625 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:49.641641 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:49.643849 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:49.646885 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.646890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.646894 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.647221 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.647694 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.647702 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.647707 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.647858 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.648390 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:49.648395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:49.648555 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 01:03:49.648699 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 01:03:49.649494 (+ 795us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 01:03:49.654968 25532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba: Bootstrap starting.
I20260806 01:03:49.657948 25532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:49.658970 25532 log.cc:824] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:49.662174 25532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba: No bootstrap required, opened a new log
I20260806 01:03:49.669349 25532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } }
I20260806 01:03:49.669660 25532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:49.669807 25532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730e2666103e480c9f13373b0109b9ba, State: Initialized, Role: FOLLOWER
I20260806 01:03:49.670408 25532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } }
I20260806 01:03:49.671449 25775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.671921 25775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:03:49.672829 25532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:03:49.683056 25792 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:03:49.683293 25792 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:03:49.682174 25442 master_runner.cc:186] Error getting master registration for 127.5.108.189:33339: OK, instance_id { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" instance_seqno: 1785978228986522 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:03:49.691833 25442 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:03:49.712634 25632 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } attempt: 5
Aug 06 01:03:49 dist-test-slave-x4qh krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
W20260806 01:03:49.722870 25680 tablet.cc:2369] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:03:49.728384 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.713246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:35013 (local address 127.0.0.1:40408)
0806 01:03:49.713496 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:49.713515 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:49.713640 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:49.713996 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:49.714000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:49.714467 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:49.714632 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.714640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.715699 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.715706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.716593 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:49.716604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.717341 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.717346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:49.717549 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.718166 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:49.718184 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:49.720312 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:49.726071 (+ 5759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.726077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.726081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.726452 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.726805 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:49.726811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:49.726815 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:49.726941 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:49.727443 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:49.727447 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:49.727590 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 01:03:49.727732 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 01:03:49.728016 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 01:03:49.728387 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:49.713306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.5.108.188:35013)
0806 01:03:49.713498 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:49.713505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:49.713597 (+ 92us) server_negotiation.cc:407] Connection header received
0806 01:03:49.713779 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:49.713784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:49.713855 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:49.713964 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:49.714779 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.715561 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.716740 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:49.717181 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:49.717325 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:49.720528 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:49.720533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:49.720536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:49.720565 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:49.725943 (+ 5378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.726564 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.726572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.726575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.726691 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:49.727044 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:49.727053 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:49.727056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:49.727311 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:49.727476 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:49.727826 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 01:03:49.728044 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 01:03:49.732725 25632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795: Bootstrap starting.
I20260806 01:03:49.735852 25632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:49.736950 25632 log.cc:824] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:49.739674 25632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795: No bootstrap required, opened a new log
W20260806 01:03:49.743013 25473 tablet.cc:2369] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:03:49.748711 25632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } }
I20260806 01:03:49.749027 25632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:49.749166 25632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50b8f01206ce404aa56402376dcb0795, State: Initialized, Role: FOLLOWER
I20260806 01:03:49.749792 25632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } }
I20260806 01:03:49.750751 25796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.751170 25796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:03:49.751915 25632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:03:49.768481 25545 master_runner.cc:263] No leader master found from master 50b8f01206ce404aa56402376dcb0795
I20260806 01:03:49.768631 25545 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 01:03:49.771109 25812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:03:49.771330 25812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:03:49.807890 25775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:03:49.808202 25775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } }
I20260806 01:03:49.809860 25775 leader_election.cc:288] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50b8f01206ce404aa56402376dcb0795 (127.5.108.189:33339), c38b5009e1494345826621e67e7469c8 (127.5.108.188:35013)
I20260806 01:03:49.811008 25705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "730e2666103e480c9f13373b0109b9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38b5009e1494345826621e67e7469c8" is_pre_election: true
I20260806 01:03:49.812201 25603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "730e2666103e480c9f13373b0109b9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50b8f01206ce404aa56402376dcb0795" is_pre_election: true
I20260806 01:03:49.812764 25603 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730e2666103e480c9f13373b0109b9ba in term 0.
I20260806 01:03:49.813519 25468 leader_election.cc:302] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50b8f01206ce404aa56402376dcb0795, 730e2666103e480c9f13373b0109b9ba; no voters:
I20260806 01:03:49.814191 25705 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730e2666103e480c9f13373b0109b9ba in term 0.
I20260806 01:03:49.815431 25775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:03:49.815654 25775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:03:49.815764 25775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:49.819209 25775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } }
I20260806 01:03:49.820441 25603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "730e2666103e480c9f13373b0109b9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50b8f01206ce404aa56402376dcb0795"
I20260806 01:03:49.820636 25603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:49.824610 25603 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730e2666103e480c9f13373b0109b9ba in term 1.
I20260806 01:03:49.825112 25468 leader_election.cc:302] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50b8f01206ce404aa56402376dcb0795, 730e2666103e480c9f13373b0109b9ba; no voters:
I20260806 01:03:49.828553 25705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "730e2666103e480c9f13373b0109b9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38b5009e1494345826621e67e7469c8"
I20260806 01:03:49.828774 25705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:49.832228 25705 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730e2666103e480c9f13373b0109b9ba in term 1.
I20260806 01:03:49.832901 25775 leader_election.cc:288] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [CANDIDATE]: Term 1 election: Requested vote from peers 50b8f01206ce404aa56402376dcb0795 (127.5.108.189:33339), c38b5009e1494345826621e67e7469c8 (127.5.108.188:35013)
I20260806 01:03:49.833288 25775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:03:49.833679 25775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [term 1 LEADER]: Becoming Leader. State: Replica: 730e2666103e480c9f13373b0109b9ba, State: Running, Role: LEADER
I20260806 01:03:49.834436 25775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } }
I20260806 01:03:49.853464 25816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 730e2666103e480c9f13373b0109b9ba. Latest consensus state: current_term: 1 leader_uuid: "730e2666103e480c9f13373b0109b9ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.854162 25816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:49.861159 25819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:03:49.863559 25819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:03:49.883575 25705 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [term 1 FOLLOWER]: Refusing update from remote peer 730e2666103e480c9f13373b0109b9ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:03:49.884826 25816 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:03:49.891980 25603 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [term 1 FOLLOWER]: Refusing update from remote peer 730e2666103e480c9f13373b0109b9ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:03:49.892867 25816 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:03:49.926415 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 730e2666103e480c9f13373b0109b9ba. Latest consensus state: current_term: 1 leader_uuid: "730e2666103e480c9f13373b0109b9ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.926788 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:03:49.927177 25796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 730e2666103e480c9f13373b0109b9ba. Latest consensus state: current_term: 1 leader_uuid: "730e2666103e480c9f13373b0109b9ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.927505 25796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:03:49.928910 25820 mvcc.cc:205] Tried to move back new op lower bound from 7315366829590728704 to 7315366829426597888. Current Snapshot: MvccSnapshot[applied={T|T < 7315366829590728704}]
I20260806 01:03:49.938269 25822 mvcc.cc:205] Tried to move back new op lower bound from 7315366829590728704 to 7315366829426597888. Current Snapshot: MvccSnapshot[applied={T|T < 7315366829590728704}]
I20260806 01:03:49.945255 25775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "730e2666103e480c9f13373b0109b9ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.945582 25775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:49.946041 25775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "730e2666103e480c9f13373b0109b9ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.946308 25775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:49.947708 25819 catalog_manager.cc:1347] Generated new cluster ID: 93df6de4ad3a48dfa7acb84b9ae43ffd
I20260806 01:03:49.947813 25819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:03:49.947981 25796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "730e2666103e480c9f13373b0109b9ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.948239 25796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:03:49.950032 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "730e2666103e480c9f13373b0109b9ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730e2666103e480c9f13373b0109b9ba" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37627 } } peers { permanent_uuid: "50b8f01206ce404aa56402376dcb0795" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 33339 } } peers { permanent_uuid: "c38b5009e1494345826621e67e7469c8" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 35013 } } }
I20260806 01:03:49.950246 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:03:49.985219 25819 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:03:49.987709 25819 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:03:50.005285 25819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 730e2666103e480c9f13373b0109b9ba: Generated new TSK 0
I20260806 01:03:50.006326 25819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:03:50.607252 25772 catalog_manager.cc:1259] Loaded cluster ID: 93df6de4ad3a48dfa7acb84b9ae43ffd
I20260806 01:03:50.607359 25772 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8: loading cluster ID for follower catalog manager: success
I20260806 01:03:50.609885 25772 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8: acquiring CA information for follower catalog manager: success
I20260806 01:03:50.611696 25772 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c38b5009e1494345826621e67e7469c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:03:50.773641 25812 catalog_manager.cc:1259] Loaded cluster ID: 93df6de4ad3a48dfa7acb84b9ae43ffd
I20260806 01:03:50.773814 25812 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795: loading cluster ID for follower catalog manager: success
I20260806 01:03:50.776458 25812 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795: acquiring CA information for follower catalog manager: success
I20260806 01:03:50.778162 25812 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 50b8f01206ce404aa56402376dcb0795: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:03:51.168154 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 25442
I20260806 01:03:51.169703 25526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:51.276005 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 25442
I20260806 01:03:51.303915 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 25545
I20260806 01:03:51.305045 25627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:51.406618 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 25545
I20260806 01:03:51.435810 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 25644
I20260806 01:03:51.436853 25736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:51.544668 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 25644
2026-08-06T01:03:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4993 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:03:51 dist-test-slave-x4qh krb5kdc[25844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:03:51 dist-test-slave-x4qh krb5kdc[25844](info): set up 2 sockets
Aug 06 01:03:51 dist-test-slave-x4qh krb5kdc[25844](info): commencing operation
krb5kdc: starting...
W20260806 01:03:52.800494 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:03:52 dist-test-slave-x4qh krb5kdc[25844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:03:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:03:53.042543 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:40761
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:34427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:40761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:03:53.365231 25862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:53.365481 25862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:53.365651 25862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:53.372748 25862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:03:53.372849 25862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:53.372895 25862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:53.372977 25862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:03:53.373064 25862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:03:53.382400 25862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:34427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:40761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:40761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:53.383771 25862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:53.385082 25862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:53.396499 25871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:53.396473 25872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:53.396620 25877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:53.397172 25862 server_base.cc:1034] running on GCE node
I20260806 01:03:53.397929 25862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:53.399118 25862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:53.400312 25862 hybrid_clock.cc:648] HybridClock initialized: now 1785978233400246 us; error 55 us; skew 500 ppm
I20260806 01:03:53.400645 25862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:53 dist-test-slave-x4qh krb5kdc[25844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978233, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:53.403522 25862 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:03:53.404953 25862 webserver.cc:460] Webserver started at http://127.5.108.190:44615/ using document root <none> and password file <none>
I20260806 01:03:53.405581 25862 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:53.405748 25862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:53.406086 25862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:03:53.408929 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "3a1cae1596954600ab8e9c2cc679f2c9"
format_stamp: "Formatted at 2026-08-06 01:03:53 on dist-test-slave-x4qh"
server_key: "819788b27a218968f8c67cb775236967"
server_key_iv: "86e1cc36053068a2f5ec92a6e92721c1"
server_key_version: "encryptionkey@0"
I20260806 01:03:53.409646 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "3a1cae1596954600ab8e9c2cc679f2c9"
format_stamp: "Formatted at 2026-08-06 01:03:53 on dist-test-slave-x4qh"
server_key: "819788b27a218968f8c67cb775236967"
server_key_iv: "86e1cc36053068a2f5ec92a6e92721c1"
server_key_version: "encryptionkey@0"
I20260806 01:03:53.414417 25862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:03:53.417689 25885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:53.418848 25862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:03:53.419021 25862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "3a1cae1596954600ab8e9c2cc679f2c9"
format_stamp: "Formatted at 2026-08-06 01:03:53 on dist-test-slave-x4qh"
server_key: "819788b27a218968f8c67cb775236967"
server_key_iv: "86e1cc36053068a2f5ec92a6e92721c1"
server_key_version: "encryptionkey@0"
I20260806 01:03:53.419210 25862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:53.440568 25862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:53.444398 25862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:53.444754 25862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:53.462675 25862 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:40761
I20260806 01:03:53.462716 25963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:40761 every 8 connection(s)
I20260806 01:03:53.463925 25862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:03:53.465277 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 25862
I20260806 01:03:53.465502 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:03:53.466161 5554 external_mini_cluster.cc:1428] Setting key abbda298500ba342d2ec569d5f09434d
I20260806 01:03:53.468007 25964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:53.480602 25964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9: Bootstrap starting.
I20260806 01:03:53.483569 25964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9: Neither blocks nor log segments found. Creating new log.
Aug 06 01:03:53 dist-test-slave-x4qh krb5kdc[25844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:53.484617 25964 log.cc:824] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:53.487296 25964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9: No bootstrap required, opened a new log
I20260806 01:03:53.491978 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:53.467937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57140 (local address 127.5.108.190:40761)
0806 01:03:53.468769 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:53.468784 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:53.468848 (+ 64us) server_negotiation.cc:407] Connection header received
0806 01:03:53.471004 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:53.471030 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:53.471535 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:53.471980 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:53.473242 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:53.475214 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:53.476501 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:53.477442 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:53.477636 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:53.484953 (+ 7317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:53.484961 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:53.484974 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:53.485019 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:53.488075 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:53.488780 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:53.488787 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:53.488793 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:53.488960 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:53.489381 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:53.489388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:53.489390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:53.489901 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:53.490283 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:53.490687 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 01:03:53.491050 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":347,"threads_started":1}
I20260806 01:03:53.494921 25964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cae1596954600ab8e9c2cc679f2c9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 40761 } }
I20260806 01:03:53.495293 25964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:53.495420 25964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1cae1596954600ab8e9c2cc679f2c9, State: Initialized, Role: FOLLOWER
I20260806 01:03:53.495649 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:03:53.495998 25964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cae1596954600ab8e9c2cc679f2c9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 40761 } }
I20260806 01:03:53.496245 25964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:03:53.496388 25964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:03:53.496549 25964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:53.498920 25964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cae1596954600ab8e9c2cc679f2c9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 40761 } }
I20260806 01:03:53.499521 25964 leader_election.cc:302] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a1cae1596954600ab8e9c2cc679f2c9; no voters:
I20260806 01:03:53.500092 25964 leader_election.cc:288] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:03:53.500232 25970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:03:53.500955 25970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1cae1596954600ab8e9c2cc679f2c9, State: Running, Role: LEADER
I20260806 01:03:53.501598 25970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cae1596954600ab8e9c2cc679f2c9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 40761 } }
I20260806 01:03:53.502019 25964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:03:53.504498 25972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a1cae1596954600ab8e9c2cc679f2c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cae1596954600ab8e9c2cc679f2c9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 40761 } } }
I20260806 01:03:53.504873 25972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:53.504592 25974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a1cae1596954600ab8e9c2cc679f2c9. Latest consensus state: current_term: 1 leader_uuid: "3a1cae1596954600ab8e9c2cc679f2c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cae1596954600ab8e9c2cc679f2c9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 40761 } } }
I20260806 01:03:53.504998 25974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:53.515915 25979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:03:53.521515 25979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:03:53.529722 25979 catalog_manager.cc:1347] Generated new cluster ID: c0e5266de1744f92bcc247c3435e22bb
I20260806 01:03:53.529822 25979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:03:53.544909 25979 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:03:53.546587 25979 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:03:53.557925 25979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a1cae1596954600ab8e9c2cc679f2c9: Generated new TSK 0
I20260806 01:03:53.558857 25979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:03:53.607198 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 25862
I20260806 01:03:53.608312 25956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:53.700824 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 25862
2026-08-06T01:03:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2145 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:03:53 dist-test-slave-x4qh krb5kdc[26006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:03:53 dist-test-slave-x4qh krb5kdc[26006](info): set up 2 sockets
Aug 06 01:03:53 dist-test-slave-x4qh krb5kdc[26006](info): commencing operation
krb5kdc: starting...
W20260806 01:03:54.964521 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:03:55 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:03:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:03:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:03:55.210611 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36209
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:03:55.539445 26023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:55.539714 26023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:55.539812 26023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:55.546471 26023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:03:55.546558 26023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:55.546602 26023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:55.546660 26023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:03:55.546702 26023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:03:55.555729 26023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:55.556905 26023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:55.558066 26023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:55.569391 26029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:55.569934 26034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:55.569990 26031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:55.570547 26023 server_base.cc:1034] running on GCE node
I20260806 01:03:55.571242 26023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:55.572659 26023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:55.573856 26023 hybrid_clock.cc:648] HybridClock initialized: now 1785978235573805 us; error 42 us; skew 500 ppm
I20260806 01:03:55.574180 26023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:55 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978235, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:55.577363 26023 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:03:55.578733 26023 webserver.cc:460] Webserver started at http://127.5.108.190:36059/ using document root <none> and password file <none>
I20260806 01:03:55.579360 26023 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:55.579542 26023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:55.579818 26023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:03:55.582644 26023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "65d00e5bf8204c268965b4610d020884"
format_stamp: "Formatted at 2026-08-06 01:03:55 on dist-test-slave-x4qh"
server_key: "9c4d5689c3510e3e211880dc0337369f"
server_key_iv: "71580501e35ab79b27a1bed4cbfc28c2"
server_key_version: "encryptionkey@0"
I20260806 01:03:55.583374 26023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "65d00e5bf8204c268965b4610d020884"
format_stamp: "Formatted at 2026-08-06 01:03:55 on dist-test-slave-x4qh"
server_key: "9c4d5689c3510e3e211880dc0337369f"
server_key_iv: "71580501e35ab79b27a1bed4cbfc28c2"
server_key_version: "encryptionkey@0"
I20260806 01:03:55.588423 26023 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 01:03:55.591820 26042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:55.593091 26023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:03:55.593227 26023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "65d00e5bf8204c268965b4610d020884"
format_stamp: "Formatted at 2026-08-06 01:03:55 on dist-test-slave-x4qh"
server_key: "9c4d5689c3510e3e211880dc0337369f"
server_key_iv: "71580501e35ab79b27a1bed4cbfc28c2"
server_key_version: "encryptionkey@0"
I20260806 01:03:55.593385 26023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:55.609556 26023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:55.613255 26023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:55.613584 26023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:55.631007 26023 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36209
I20260806 01:03:55.631037 26122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36209 every 8 connection(s)
I20260806 01:03:55.632611 26023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:03:55.636273 26123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:55.636662 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 26023
I20260806 01:03:55.636909 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:03:55.637522 5554 external_mini_cluster.cc:1428] Setting key b6677ca3e97b24140b32aaf6291d1cb5
I20260806 01:03:55.649912 26123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Bootstrap starting.
Aug 06 01:03:55 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:55.653478 26123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:55.654844 26123 log.cc:824] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:55.658066 26123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: No bootstrap required, opened a new log
I20260806 01:03:55.661593 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:55.639511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53716 (local address 127.5.108.190:36209)
0806 01:03:55.640321 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:55.640331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:55.640374 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:03:55.642080 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:55.642102 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:55.642463 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:55.642778 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:55.644550 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:55.645874 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:55.647761 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:55.648396 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:55.648564 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:55.653389 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:55.653400 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:55.653418 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:55.653488 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:55.657152 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:55.657852 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:55.657859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:55.657866 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:55.658069 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:55.658460 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:55.658464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:55.658467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:55.659169 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:55.659536 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:55.659889 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 01:03:55.660358 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":375,"threads_started":1}
I20260806 01:03:55.666980 26123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:03:55.667277 26123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:55.667347 26123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d00e5bf8204c268965b4610d020884, State: Initialized, Role: FOLLOWER
I20260806 01:03:55.667877 26123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:03:55.668088 26123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:03:55.668184 26123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:03:55.668303 26123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:55.670665 26123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:03:55.671082 26123 leader_election.cc:302] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d00e5bf8204c268965b4610d020884; no voters:
I20260806 01:03:55.671816 26123 leader_election.cc:288] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:03:55.671964 26129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:03:55.672690 26129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 1 LEADER]: Becoming Leader. State: Replica: 65d00e5bf8204c268965b4610d020884, State: Running, Role: LEADER
I20260806 01:03:55.673195 26129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:03:55.673863 26123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:03:55.676052 26131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d00e5bf8204c268965b4610d020884. Latest consensus state: current_term: 1 leader_uuid: "65d00e5bf8204c268965b4610d020884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } } }
I20260806 01:03:55.676077 26130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65d00e5bf8204c268965b4610d020884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } } }
I20260806 01:03:55.676458 26131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:55.676458 26130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:55.681825 26138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:03:55.686991 26138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:03:55.697000 26138 catalog_manager.cc:1347] Generated new cluster ID: 4d397331e5634217a8ff8dc0bce1a2ff
I20260806 01:03:55.697125 26138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:03:55.724723 26138 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:03:55.726022 26138 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:03:55.735025 26138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Generated new TSK 0
I20260806 01:03:55.735831 26138 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:03:55.814613 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36209
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:03:56.153635 26157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:56.153894 26157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:56.153980 26157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:56.160630 26157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:56.160718 26157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:56.160913 26157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:03:56.169780 26157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:56.170883 26157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:56.172259 26157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:56.185509 26166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:56.185425 26167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:56.185478 26170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:56.186228 26157 server_base.cc:1034] running on GCE node
I20260806 01:03:56.186810 26157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:56.187812 26157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:56.189047 26157 hybrid_clock.cc:648] HybridClock initialized: now 1785978236188992 us; error 46 us; skew 500 ppm
I20260806 01:03:56.189343 26157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:56 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978236, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:56.192566 26157 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:03:56.193792 26157 webserver.cc:460] Webserver started at http://127.5.108.129:40801/ using document root <none> and password file <none>
I20260806 01:03:56.194375 26157 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:56.194469 26157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:56.194710 26157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:03:56.197353 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "b1fed31bf91448a3b44d4b72c39e0395"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "593d310436f4d0ce9df069df24d63077"
server_key_iv: "d6efebc00652354302f5ec1783d8e311"
server_key_version: "encryptionkey@0"
I20260806 01:03:56.198000 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "b1fed31bf91448a3b44d4b72c39e0395"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "593d310436f4d0ce9df069df24d63077"
server_key_iv: "d6efebc00652354302f5ec1783d8e311"
server_key_version: "encryptionkey@0"
I20260806 01:03:56.202659 26157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260806 01:03:56.205785 26180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:56.207079 26157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:03:56.207216 26157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "b1fed31bf91448a3b44d4b72c39e0395"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "593d310436f4d0ce9df069df24d63077"
server_key_iv: "d6efebc00652354302f5ec1783d8e311"
server_key_version: "encryptionkey@0"
I20260806 01:03:56.207437 26157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:56.239326 26157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:56.243669 26157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:56.243894 26157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:56.244778 26157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:56.245981 26157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:56.246048 26157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:56.246140 26157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:56.246179 26157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:56.278081 26157 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:37477
I20260806 01:03:56.278160 26347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:37477 every 8 connection(s)
I20260806 01:03:56.279376 26157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:03:56.283524 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 26157
I20260806 01:03:56.283743 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:03:56.284469 5554 external_mini_cluster.cc:1428] Setting key 73171b2e1cdefae4b7da43f50efc1a5d
Aug 06 01:03:56 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978236, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:56.299448 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:56.282066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:35589 (local address 127.5.108.190:36209)
0806 01:03:56.282511 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:56.282515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:56.283493 (+ 978us) server_negotiation.cc:407] Connection header received
0806 01:03:56.285505 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:56.285511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:56.285631 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:56.285756 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:56.287265 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:56.288030 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:56.289515 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:56.289929 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:56.290075 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:56.293339 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:56.293343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:56.293347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:56.293378 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:56.295949 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:56.296813 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:56.296819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:56.296826 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:56.296950 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:56.297421 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:56.297424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:56.297426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:56.297852 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:56.298025 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:56.298845 (+ 820us) server_negotiation.cc:299] Negotiation successful
0806 01:03:56.299142 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":244,"threads_started":1}
I20260806 01:03:56.300110 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:56.282604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:35589)
0806 01:03:56.283298 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:56.283340 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:56.285162 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:56.285952 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:56.285977 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:56.286551 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:56.287100 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:56.287116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:56.288192 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:56.288196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:56.289361 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:56.289373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:56.290166 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:56.290175 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:56.290414 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:56.291060 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:56.291085 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:56.293130 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:56.296086 (+ 2956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:56.296094 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:56.296107 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:56.296685 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:56.297063 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:56.297068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:56.297072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:56.297260 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:56.298014 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:56.298023 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:56.298462 (+ 439us) client_negotiation.cc:769] Sending connection context
0806 01:03:56.298695 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 01:03:56.299062 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":259,"threads_started":1}
I20260806 01:03:56.301651 26351 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36209
I20260806 01:03:56.301966 26351 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:56.302836 26351 heartbeater.cc:511] Master 127.5.108.190:36209 requested a full tablet report, sending...
I20260806 01:03:56.305164 26067 ts_manager.cc:194] Registered new tserver with Master: b1fed31bf91448a3b44d4b72c39e0395 (127.5.108.129:37477)
I20260806 01:03:56.306593 26067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:35589
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:03:56.345778 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36209
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:03:56.669551 26363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:56.669847 26363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:56.670017 26363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:56.676962 26363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:56.677091 26363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:56.677271 26363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:03:56.686661 26363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:56.687848 26363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:56.689103 26363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:56.701648 26368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:56.701678 26369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:56.701745 26373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:56.702687 26363 server_base.cc:1034] running on GCE node
I20260806 01:03:56.703169 26363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:56.704018 26363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:56.705214 26363 hybrid_clock.cc:648] HybridClock initialized: now 1785978236705157 us; error 47 us; skew 500 ppm
I20260806 01:03:56.705513 26363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:56 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978236, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:56.708493 26363 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:03:56.709758 26363 webserver.cc:460] Webserver started at http://127.5.108.130:37461/ using document root <none> and password file <none>
I20260806 01:03:56.710435 26363 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:56.710532 26363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:56.710785 26363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:03:56.713585 26363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "4c3251d466e04c2eb532c2e4841e017f"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "0ef9acc72df4cbae7c31a8a0224c7216"
server_key_iv: "51bd7d0e0df1b5d32c0d1d88aa1a02b4"
server_key_version: "encryptionkey@0"
I20260806 01:03:56.714267 26363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "4c3251d466e04c2eb532c2e4841e017f"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "0ef9acc72df4cbae7c31a8a0224c7216"
server_key_iv: "51bd7d0e0df1b5d32c0d1d88aa1a02b4"
server_key_version: "encryptionkey@0"
I20260806 01:03:56.719332 26363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260806 01:03:56.722664 26383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:56.723960 26363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:03:56.724129 26363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "4c3251d466e04c2eb532c2e4841e017f"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "0ef9acc72df4cbae7c31a8a0224c7216"
server_key_iv: "51bd7d0e0df1b5d32c0d1d88aa1a02b4"
server_key_version: "encryptionkey@0"
I20260806 01:03:56.724328 26363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:56.742662 26363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:56.746934 26363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:56.747205 26363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:56.748248 26363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:56.749612 26363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:56.749717 26363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:56.749869 26363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:56.749943 26363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:56.782774 26363 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:40545
I20260806 01:03:56.782826 26540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:40545 every 8 connection(s)
I20260806 01:03:56.784101 26363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:03:56.788736 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 26363
I20260806 01:03:56.788924 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:03:56.789393 5554 external_mini_cluster.cc:1428] Setting key 24d386ed07dee184561b828a0866583c
Aug 06 01:03:56 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978236, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:56.805052 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:56.786683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:35279 (local address 127.5.108.190:36209)
0806 01:03:56.786860 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:56.786864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:56.788033 (+ 1169us) server_negotiation.cc:407] Connection header received
0806 01:03:56.790015 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:56.790019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:56.790087 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:56.790211 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:56.791821 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:56.792467 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:56.794839 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:56.795222 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:56.795310 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:56.799421 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:56.799435 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:56.799439 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:56.799472 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:56.802039 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:56.802739 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:56.802744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:56.802749 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:56.802878 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:56.803324 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:56.803327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:56.803329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:56.803629 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:56.803822 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:56.804479 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 01:03:56.804708 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:03:56.805981 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:56.787142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:35279)
0806 01:03:56.787876 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:56.787911 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:56.789713 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:56.790463 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:56.790471 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:56.790945 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:56.791645 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:56.791671 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:56.792617 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:56.792624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:56.794693 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:56.794712 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:56.795467 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:56.795477 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:56.795798 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:56.796513 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:56.796550 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:56.799148 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:56.802187 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:56.802196 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:56.802211 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:56.802608 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:56.802993 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:56.802998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:56.803002 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:56.803195 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:56.803788 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:56.803796 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:56.804103 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 01:03:56.804381 (+ 278us) client_negotiation.cc:240] Negotiation successful
0806 01:03:56.804880 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":308,"threads_started":1}
I20260806 01:03:56.807451 26541 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36209
I20260806 01:03:56.807804 26541 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:56.808661 26541 heartbeater.cc:511] Master 127.5.108.190:36209 requested a full tablet report, sending...
I20260806 01:03:56.810320 26067 ts_manager.cc:194] Registered new tserver with Master: 4c3251d466e04c2eb532c2e4841e017f (127.5.108.130:40545)
I20260806 01:03:56.811657 26067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:35279
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:03:56.852402 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36209
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:03:57.192229 26546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:57.192476 26546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:57.192557 26546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:57.199375 26546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:57.199489 26546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:57.199604 26546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:03:57.208525 26546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:57.209587 26546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:57.210927 26546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:57.223644 26553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:57.223680 26556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:57.223749 26560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:57.224865 26546 server_base.cc:1034] running on GCE node
I20260806 01:03:57.225452 26546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:57.226313 26546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:57.227496 26546 hybrid_clock.cc:648] HybridClock initialized: now 1785978237227446 us; error 53 us; skew 500 ppm
I20260806 01:03:57.227797 26546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:57 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978237, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:57.230844 26546 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:03:57.232219 26546 webserver.cc:460] Webserver started at http://127.5.108.131:34253/ using document root <none> and password file <none>
I20260806 01:03:57.232842 26546 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:57.232975 26546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:57.233253 26546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:03:57.235950 26546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "cd734bb0b7df4d599266b27199dce349"
format_stamp: "Formatted at 2026-08-06 01:03:57 on dist-test-slave-x4qh"
server_key: "3fbdfd36b5ae65fe6dd47d99c5052b31"
server_key_iv: "a6b0bbdeeb3ad67a59cc2343d446593c"
server_key_version: "encryptionkey@0"
I20260806 01:03:57.236708 26546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "cd734bb0b7df4d599266b27199dce349"
format_stamp: "Formatted at 2026-08-06 01:03:57 on dist-test-slave-x4qh"
server_key: "3fbdfd36b5ae65fe6dd47d99c5052b31"
server_key_iv: "a6b0bbdeeb3ad67a59cc2343d446593c"
server_key_version: "encryptionkey@0"
I20260806 01:03:57.241477 26546 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:03:57.244588 26571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:57.245779 26546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:03:57.245935 26546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "cd734bb0b7df4d599266b27199dce349"
format_stamp: "Formatted at 2026-08-06 01:03:57 on dist-test-slave-x4qh"
server_key: "3fbdfd36b5ae65fe6dd47d99c5052b31"
server_key_iv: "a6b0bbdeeb3ad67a59cc2343d446593c"
server_key_version: "encryptionkey@0"
I20260806 01:03:57.246111 26546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:57.263921 26546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:57.268172 26546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:57.268447 26546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:57.269348 26546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:57.270622 26546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:57.270725 26546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:57.270876 26546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:57.270952 26546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:57.304569 26546 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:33523
I20260806 01:03:57.304649 26748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:33523 every 8 connection(s)
I20260806 01:03:57.305802 26546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:03:57.307961 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 26546
I20260806 01:03:57.308162 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:03:57.308761 5554 external_mini_cluster.cc:1428] Setting key 1597d71c9f844fd447fe57b3ef2f011b
I20260806 01:03:57.310001 26351 heartbeater.cc:503] Master 127.5.108.190:36209 was elected leader, sending a full tablet report...
Aug 06 01:03:57 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978237, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:57.325631 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.308281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:46793 (local address 127.5.108.190:36209)
0806 01:03:57.308793 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:57.308799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:57.309806 (+ 1007us) server_negotiation.cc:407] Connection header received
0806 01:03:57.312605 (+ 2799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:57.312608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:57.312677 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:57.312758 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:57.314087 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.314828 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.316196 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.316544 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.316630 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.319666 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:57.319671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:57.319674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:57.319705 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:57.322503 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:57.323418 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:57.323423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:57.323428 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:57.323588 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:57.324090 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:57.324095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:57.324097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:57.324470 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:57.324593 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:57.325186 (+ 593us) server_negotiation.cc:299] Negotiation successful
0806 01:03:57.325361 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":253,"threads_started":1}
I20260806 01:03:57.326418 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.308830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:46793)
0806 01:03:57.309656 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:57.309699 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:57.312362 (+ 2663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:57.312944 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:57.312950 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:57.313441 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:57.313947 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.313962 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.314938 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.314941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.316066 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.316076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.316712 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.316716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.316880 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.317456 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:57.317479 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:57.319473 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:57.322643 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:57.322653 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:57.322669 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:57.323257 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:57.323730 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:57.323733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:57.323742 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:57.323960 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:57.324594 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:57.324600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:57.324874 (+ 274us) client_negotiation.cc:769] Sending connection context
0806 01:03:57.325078 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 01:03:57.325419 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":361,"threads_started":1}
I20260806 01:03:57.327744 26751 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36209
I20260806 01:03:57.328001 26751 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:57.328717 26751 heartbeater.cc:511] Master 127.5.108.190:36209 requested a full tablet report, sending...
I20260806 01:03:57.330165 26066 ts_manager.cc:194] Registered new tserver with Master: cd734bb0b7df4d599266b27199dce349 (127.5.108.131:33523)
I20260806 01:03:57.331032 26066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:46793
I20260806 01:03:57.336301 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:03:57.353345 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.340696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.5.108.190:36209)
0806 01:03:57.340850 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:57.340855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:57.341317 (+ 462us) server_negotiation.cc:407] Connection header received
0806 01:03:57.341501 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:57.341507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:57.341572 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:57.341671 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:57.343016 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.343761 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.345058 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.345516 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.345703 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.348187 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:57.348193 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:57.348198 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:57.348235 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:57.350853 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:57.351512 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:57.351517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:57.351521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:57.351637 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:57.351977 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:57.351983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:57.351988 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:57.352281 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:57.352513 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:57.352843 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 01:03:57.353046 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:03:57.357012 26066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:03:57.363603 26066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:03:57.385777 26775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.376662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.5.108.129:37477)
0806 01:03:57.378704 (+ 2042us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:57.378710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:57.378730 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:03:57.378804 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:57.378808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:57.378953 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:57.379143 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:57.379830 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.381727 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.383795 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.384727 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.384844 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.385004 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:57.385391 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 01:03:57.385510 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1903,"thread_start_us":1755,"threads_started":1}
I20260806 01:03:57.385906 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.376458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37477 (local address 127.0.0.1:39180)
0806 01:03:57.377601 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:57.377636 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:57.377767 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:57.379141 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:57.379146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:57.379169 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:57.379625 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.379640 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.381846 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.381853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.383578 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.383593 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.384929 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.384936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.385153 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.385214 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 01:03:57.385342 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 01:03:57.385549 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":3,"thread_start_us":875,"threads_started":1}
I20260806 01:03:57.387755 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.375951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49016 (local address 127.5.108.131:33523)
0806 01:03:57.377250 (+ 1299us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:57.377262 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:57.377308 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:03:57.377425 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:57.377436 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:57.377697 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:57.377982 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:57.378755 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.380907 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.385291 (+ 4384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.386682 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.386818 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.387024 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:57.387255 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 01:03:57.387446 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1098,"thread_start_us":967,"threads_started":1}
I20260806 01:03:57.387758 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.375704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:33523 (local address 127.0.0.1:49016)
0806 01:03:57.376555 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:57.376572 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:57.376677 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:57.377984 (+ 1307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:57.377994 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:57.378028 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:57.378467 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.378488 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.381039 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.381046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.383544 (+ 2498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.383555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.386884 (+ 3329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.386891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.387056 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.387086 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:03:57.387182 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 01:03:57.387310 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":3,"thread_start_us":362,"threads_started":1}
I20260806 01:03:57.388562 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.375023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:40545 (local address 127.0.0.1:58796)
0806 01:03:57.375826 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:57.375866 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:57.376053 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:57.381284 (+ 5231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:57.381288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:57.381312 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:57.381710 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.381722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.383546 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.383550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.385118 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.385131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.387789 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.387796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.387969 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.388006 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:03:57.388066 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 01:03:57.388197 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":3,"thread_start_us":407,"threads_started":1}
I20260806 01:03:57.389451 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.375201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58796 (local address 127.5.108.130:40545)
0806 01:03:57.380756 (+ 5555us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:57.380763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:57.380788 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:03:57.380876 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:57.380884 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:57.381096 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:57.381271 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:57.381914 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.383352 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.386266 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.387540 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.388874 (+ 1334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.389029 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:57.389079 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 01:03:57.389216 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5396,"thread_start_us":5282,"threads_started":1}
I20260806 01:03:57.390129 26255 tablet_service.cc:1467] Processing CreateTablet for tablet ef23fd4a133d4670a89aa03b397fa916 (DEFAULT_TABLE table=test-table [id=a32a28874617460e92a6900d22ac3a8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:03:57.391762 26255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef23fd4a133d4670a89aa03b397fa916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:57.391448 26643 tablet_service.cc:1467] Processing CreateTablet for tablet ef23fd4a133d4670a89aa03b397fa916 (DEFAULT_TABLE table=test-table [id=a32a28874617460e92a6900d22ac3a8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:03:57.392336 26448 tablet_service.cc:1467] Processing CreateTablet for tablet ef23fd4a133d4670a89aa03b397fa916 (DEFAULT_TABLE table=test-table [id=a32a28874617460e92a6900d22ac3a8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:03:57.392714 26643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef23fd4a133d4670a89aa03b397fa916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:57.393280 26448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef23fd4a133d4670a89aa03b397fa916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:03:57.404034 26777 tablet_bootstrap.cc:492] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: Bootstrap starting.
I20260806 01:03:57.405910 26778 tablet_bootstrap.cc:492] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: Bootstrap starting.
I20260806 01:03:57.407639 26777 tablet_bootstrap.cc:654] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:57.408890 26777 log.cc:824] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:57.409227 26778 tablet_bootstrap.cc:654] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:57.410584 26778 log.cc:824] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:57.411500 26779 tablet_bootstrap.cc:492] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: Bootstrap starting.
I20260806 01:03:57.412204 26777 tablet_bootstrap.cc:492] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: No bootstrap required, opened a new log
I20260806 01:03:57.412467 26777 ts_tablet_manager.cc:1397] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260806 01:03:57.413136 26778 tablet_bootstrap.cc:492] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: No bootstrap required, opened a new log
I20260806 01:03:57.413369 26778 ts_tablet_manager.cc:1397] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 01:03:57.414475 26779 tablet_bootstrap.cc:654] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: Neither blocks nor log segments found. Creating new log.
I20260806 01:03:57.415522 26779 log.cc:824] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:57.419111 26779 tablet_bootstrap.cc:492] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: No bootstrap required, opened a new log
I20260806 01:03:57.419353 26779 ts_tablet_manager.cc:1397] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 01:03:57.420835 26778 raft_consensus.cc:348] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } }
I20260806 01:03:57.421182 26778 raft_consensus.cc:374] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:57.421300 26778 raft_consensus.cc:729] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1fed31bf91448a3b44d4b72c39e0395, State: Initialized, Role: FOLLOWER
I20260806 01:03:57.421868 26778 consensus_queue.cc:260] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } }
I20260806 01:03:57.421983 26777 raft_consensus.cc:348] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } }
I20260806 01:03:57.422443 26777 raft_consensus.cc:374] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:57.422569 26777 raft_consensus.cc:729] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c3251d466e04c2eb532c2e4841e017f, State: Initialized, Role: FOLLOWER
I20260806 01:03:57.423345 26778 ts_tablet_manager.cc:1428] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260806 01:03:57.423913 26777 consensus_queue.cc:260] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } }
I20260806 01:03:57.425019 26541 heartbeater.cc:503] Master 127.5.108.190:36209 was elected leader, sending a full tablet report...
I20260806 01:03:57.425688 26777 ts_tablet_manager.cc:1428] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260806 01:03:57.429322 26779 raft_consensus.cc:348] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } }
I20260806 01:03:57.429665 26779 raft_consensus.cc:374] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:03:57.429790 26779 raft_consensus.cc:729] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd734bb0b7df4d599266b27199dce349, State: Initialized, Role: FOLLOWER
I20260806 01:03:57.430280 26779 consensus_queue.cc:260] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } }
I20260806 01:03:57.431149 26751 heartbeater.cc:503] Master 127.5.108.190:36209 was elected leader, sending a full tablet report...
I20260806 01:03:57.431748 26779 ts_tablet_manager.cc:1428] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
W20260806 01:03:57.531795 26354 tablet.cc:2369] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:03:57.536115 26542 tablet.cc:2369] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:03:57.557499 26752 tablet.cc:2369] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:03:57.848409 26788 raft_consensus.cc:482] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:03:57.848724 26788 raft_consensus.cc:504] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } }
I20260806 01:03:57.850585 26788 leader_election.cc:288] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c3251d466e04c2eb532c2e4841e017f (127.5.108.130:40545), cd734bb0b7df4d599266b27199dce349 (127.5.108.131:33523)
I20260806 01:03:57.859584 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.851097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:37903 (local address 127.5.108.130:40545)
0806 01:03:57.851273 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:57.851279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:57.852037 (+ 758us) server_negotiation.cc:407] Connection header received
0806 01:03:57.852245 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:57.852250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:57.852315 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:57.852426 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:57.853251 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.854259 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.857173 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.858382 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.858521 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.858590 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:57.859176 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 01:03:57.859314 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 01:03:57.859583 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.851050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:55495 (local address 127.5.108.131:33523)
0806 01:03:57.851284 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:57.851290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:57.852036 (+ 746us) server_negotiation.cc:407] Connection header received
0806 01:03:57.852228 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:57.852232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:57.852309 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:57.852425 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:03:57.853280 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.854260 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.857175 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.858365 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.858534 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.858595 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:57.859176 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 01:03:57.859305 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260806 01:03:57.859848 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.851005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:33523 (local address 127.5.108.129:55495)
0806 01:03:57.851904 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:57.851924 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:57.852070 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:57.852468 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:57.852473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:57.852521 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:57.853126 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.853146 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.854400 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.854405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.856985 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.857000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.858670 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.858674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.858926 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.859013 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 01:03:57.859111 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 01:03:57.859304 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":354,"spinlock_wait_cycles":15872,"thread_start_us":504,"threads_started":1}
I20260806 01:03:57.859890 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.850856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:40545 (local address 127.5.108.129:37903)
0806 01:03:57.851904 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:57.851924 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:57.852110 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:57.852468 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:57.852473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:57.852521 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:03:57.853054 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.853075 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.854400 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.854405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.856985 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:57.857000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.858661 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.858667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:57.858922 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.859014 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 01:03:57.859113 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 01:03:57.859304 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":275,"spinlock_wait_cycles":44288,"thread_start_us":671,"threads_started":1}
I20260806 01:03:57.861076 26681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef23fd4a133d4670a89aa03b397fa916" candidate_uuid: "b1fed31bf91448a3b44d4b72c39e0395" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd734bb0b7df4d599266b27199dce349" is_pre_election: true
I20260806 01:03:57.861076 26478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef23fd4a133d4670a89aa03b397fa916" candidate_uuid: "b1fed31bf91448a3b44d4b72c39e0395" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c3251d466e04c2eb532c2e4841e017f" is_pre_election: true
I20260806 01:03:57.861609 26681 raft_consensus.cc:2454] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1fed31bf91448a3b44d4b72c39e0395 in term 0.
I20260806 01:03:57.861608 26478 raft_consensus.cc:2454] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1fed31bf91448a3b44d4b72c39e0395 in term 0.
I20260806 01:03:57.862290 26182 leader_election.cc:302] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1fed31bf91448a3b44d4b72c39e0395, cd734bb0b7df4d599266b27199dce349; no voters:
I20260806 01:03:57.862721 26788 raft_consensus.cc:2781] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:03:57.862895 26788 raft_consensus.cc:482] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:03:57.863003 26788 raft_consensus.cc:3037] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:57.865238 26788 raft_consensus.cc:504] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } }
I20260806 01:03:57.865815 26788 leader_election.cc:288] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [CANDIDATE]: Term 1 election: Requested vote from peers 4c3251d466e04c2eb532c2e4841e017f (127.5.108.130:40545), cd734bb0b7df4d599266b27199dce349 (127.5.108.131:33523)
I20260806 01:03:57.866329 26478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef23fd4a133d4670a89aa03b397fa916" candidate_uuid: "b1fed31bf91448a3b44d4b72c39e0395" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c3251d466e04c2eb532c2e4841e017f"
I20260806 01:03:57.866329 26681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef23fd4a133d4670a89aa03b397fa916" candidate_uuid: "b1fed31bf91448a3b44d4b72c39e0395" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd734bb0b7df4d599266b27199dce349"
I20260806 01:03:57.866546 26681 raft_consensus.cc:3037] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:57.866550 26478 raft_consensus.cc:3037] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:03:57.869653 26478 raft_consensus.cc:2454] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1fed31bf91448a3b44d4b72c39e0395 in term 1.
I20260806 01:03:57.869642 26681 raft_consensus.cc:2454] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1fed31bf91448a3b44d4b72c39e0395 in term 1.
I20260806 01:03:57.870167 26182 leader_election.cc:302] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1fed31bf91448a3b44d4b72c39e0395, cd734bb0b7df4d599266b27199dce349; no voters:
I20260806 01:03:57.870499 26788 raft_consensus.cc:2781] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:03:57.870941 26788 raft_consensus.cc:686] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 1 LEADER]: Becoming Leader. State: Replica: b1fed31bf91448a3b44d4b72c39e0395, State: Running, Role: LEADER
I20260806 01:03:57.871445 26788 consensus_queue.cc:237] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } }
I20260806 01:03:57.876204 26067 catalog_manager.cc:5495] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1fed31bf91448a3b44d4b72c39e0395 (127.5.108.129). New cstate: current_term: 1 leader_uuid: "b1fed31bf91448a3b44d4b72c39e0395" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } health_report { overall_health: HEALTHY } } }
I20260806 01:03:57.949621 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.944274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.5.108.129:37477)
0806 01:03:57.944745 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:57.944749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:57.944868 (+ 119us) server_negotiation.cc:407] Connection header received
0806 01:03:57.945048 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:57.945051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:57.945128 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:57.945264 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:03:57.945652 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.946261 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.947601 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:57.947962 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:57.948049 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:57.948503 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:03:57.948967 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:03:57.949074 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:57.949191 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 01:03:57.949300 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":265,"threads_started":1}
I20260806 01:03:57.956842 26681 raft_consensus.cc:1261] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [term 1 FOLLOWER]: Refusing update from remote peer b1fed31bf91448a3b44d4b72c39e0395: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:03:57.957049 26478 raft_consensus.cc:1261] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [term 1 FOLLOWER]: Refusing update from remote peer b1fed31bf91448a3b44d4b72c39e0395: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:03:57.957726 26797 consensus_queue.cc:1035] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:03:57.958060 26788 consensus_queue.cc:1035] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:03:57.967890 26808 mvcc.cc:205] Tried to move back new op lower bound from 7315366862662651904 to 7315366862325186560. Current Snapshot: MvccSnapshot[applied={T|T < 7315366862662651904}]
I20260806 01:03:57.969901 26818 mvcc.cc:205] Tried to move back new op lower bound from 7315366862662651904 to 7315366862325186560. Current Snapshot: MvccSnapshot[applied={T|T < 7315366862662651904}]
I20260806 01:03:57.980522 26066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:03:57.980818 26066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:03:57.984265 26066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Sending DeleteTablet for 3 replicas of tablet ef23fd4a133d4670a89aa03b397fa916
I20260806 01:03:57.985419 26448 tablet_service.cc:1514] Processing DeleteTablet for tablet ef23fd4a133d4670a89aa03b397fa916 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:03:57 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:58796
I20260806 01:03:57.985569 26255 tablet_service.cc:1514] Processing DeleteTablet for tablet ef23fd4a133d4670a89aa03b397fa916 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:03:57 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:39180
I20260806 01:03:57.985764 26643 tablet_service.cc:1514] Processing DeleteTablet for tablet ef23fd4a133d4670a89aa03b397fa916 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:03:57 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:49016
I20260806 01:03:57.986624 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 26157
I20260806 01:03:58.001124 26830 tablet_replica.cc:330] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: stopping tablet replica
I20260806 01:03:58.001856 26830 raft_consensus.cc:2229] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:03:58.002465 26830 raft_consensus.cc:2258] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:03:58.004655 26830 ts_tablet_manager.cc:1905] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:03:58.012940 26830 ts_tablet_manager.cc:1918] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:03:58.013217 26830 log.cc:1196] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/ef23fd4a133d4670a89aa03b397fa916
I20260806 01:03:58.013845 26830 ts_tablet_manager.cc:1939] T ef23fd4a133d4670a89aa03b397fa916 P cd734bb0b7df4d599266b27199dce349: Deleting consensus metadata
I20260806 01:03:58.015126 26828 tablet_replica.cc:330] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: stopping tablet replica
I20260806 01:03:58.015628 26828 raft_consensus.cc:2229] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:03:58.016011 26045 catalog_manager.cc:4841] TS cd734bb0b7df4d599266b27199dce349 (127.5.108.131:33523): tablet ef23fd4a133d4670a89aa03b397fa916 (table test-table [id=a32a28874617460e92a6900d22ac3a8b]) successfully deleted
I20260806 01:03:58.016188 26828 raft_consensus.cc:2258] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:03:58.020159 26828 ts_tablet_manager.cc:1905] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:03:58 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:03:58.025200 26829 tablet_replica.cc:330] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: stopping tablet replica
I20260806 01:03:58.025612 26829 raft_consensus.cc:2229] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:03:58.026255 26829 raft_consensus.cc:2258] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:03:58.027115 26828 ts_tablet_manager.cc:1918] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:03:58.027340 26828 log.cc:1196] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/ef23fd4a133d4670a89aa03b397fa916
I20260806 01:03:58.027894 26828 ts_tablet_manager.cc:1939] T ef23fd4a133d4670a89aa03b397fa916 P 4c3251d466e04c2eb532c2e4841e017f: Deleting consensus metadata
I20260806 01:03:58.028863 26829 ts_tablet_manager.cc:1905] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:03:58.029733 26047 catalog_manager.cc:4841] TS 4c3251d466e04c2eb532c2e4841e017f (127.5.108.130:40545): tablet ef23fd4a133d4670a89aa03b397fa916 (table test-table [id=a32a28874617460e92a6900d22ac3a8b]) successfully deleted
I20260806 01:03:58.034190 26829 ts_tablet_manager.cc:1918] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:03:58.034373 26829 log.cc:1196] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/ef23fd4a133d4670a89aa03b397fa916
I20260806 01:03:58.035036 26829 ts_tablet_manager.cc:1939] T ef23fd4a133d4670a89aa03b397fa916 P b1fed31bf91448a3b44d4b72c39e0395: Deleting consensus metadata
I20260806 01:03:58.035789 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:57.987816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.5.108.129:37477)
0806 01:03:57.988051 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:57.988059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:58.011873 (+ 23814us) server_negotiation.cc:407] Connection header received
0806 01:03:58.012308 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:58.012314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:58.012384 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:58.012450 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:58.016564 (+ 4114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:58.017471 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:58.019544 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:58.020253 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:58.020367 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:58.027680 (+ 7313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:58.027694 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:58.027709 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:58.027759 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:58.032158 (+ 4399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:58.033033 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:58.033041 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:58.033044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:58.033283 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:58.033790 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:58.033795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:58.033798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:58.034240 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:58.034551 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:58.035052 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 01:03:58.035315 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:03:58.036712 26341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:58.036798 26045 catalog_manager.cc:4841] TS b1fed31bf91448a3b44d4b72c39e0395 (127.5.108.129:37477): tablet ef23fd4a133d4670a89aa03b397fa916 (table test-table [id=a32a28874617460e92a6900d22ac3a8b]) successfully deleted
I20260806 01:03:58.130241 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 26157
I20260806 01:03:58.160840 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 26363
Aug 06 01:03:58 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:03:58.180083 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:58.161707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.5.108.130:40545)
0806 01:03:58.161902 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:58.161907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:58.161926 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:03:58.162053 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:58.162057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:58.162115 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:58.162222 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:58.163462 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:58.164171 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:58.165537 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:58.165944 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:58.166076 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:58.174373 (+ 8297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:58.174386 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:58.174401 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:58.174434 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:58.177238 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:58.177893 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:58.177908 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:58.177911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:58.178033 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:58.178408 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:58.178413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:58.178416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:58.178789 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:58.179092 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:58.179568 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 01:03:58.179797 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:03:58.180822 26536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:58.266819 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 26363
I20260806 01:03:58.292481 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 26546
Aug 06 01:03:58 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:03:58.308540 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:58.293379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49022 (local address 127.5.108.131:33523)
0806 01:03:58.293565 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:58.293571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:58.293591 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:03:58.293695 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:58.293699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:58.293797 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:58.293913 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:58.295240 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:58.296013 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:58.297337 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:58.297710 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:58.297847 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:58.302195 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:58.302211 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:58.302227 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:58.302264 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:58.305628 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:58.306313 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:58.306328 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:58.306331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:58.306524 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:58.306898 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:58.306903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:58.306907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:58.307299 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:58.307616 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:58.307885 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 01:03:58.308118 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:03:58.309257 26743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:58.395877 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 26546
I20260806 01:03:58.420893 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 26023
I20260806 01:03:58.422108 26114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:03:58.526739 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 26023
I20260806 01:03:58.555590 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36209
--webserver_interface=127.5.108.190
--webserver_port=36059
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:03:58.883431 26841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:58.883755 26841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:58.883925 26841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:58.890853 26841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:03:58.890939 26841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:58.891053 26841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:58.891185 26841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:03:58.891273 26841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:03:58.900408 26841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=36059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:58.901585 26841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:58.902845 26841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:58.928339 26852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:58.928380 26851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:58.928341 26857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:58.929224 26841 server_base.cc:1034] running on GCE node
I20260806 01:03:58.930073 26841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:58.931670 26841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:58.932916 26841 hybrid_clock.cc:648] HybridClock initialized: now 1785978238932844 us; error 52 us; skew 500 ppm
I20260806 01:03:58.933323 26841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:58 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978238, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:58.941881 26841 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:03:58.943629 26841 webserver.cc:460] Webserver started at http://127.5.108.190:36059/ using document root <none> and password file <none>
I20260806 01:03:58.944461 26841 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:58.944605 26841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:58.950610 26841 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 01:03:58.953420 26864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:58.954666 26841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:03:58.954841 26841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "65d00e5bf8204c268965b4610d020884"
format_stamp: "Formatted at 2026-08-06 01:03:55 on dist-test-slave-x4qh"
server_key: "9c4d5689c3510e3e211880dc0337369f"
server_key_iv: "71580501e35ab79b27a1bed4cbfc28c2"
server_key_version: "encryptionkey@0"
I20260806 01:03:58.955994 26841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:58.971971 26841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:58.976006 26841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:58.976240 26841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:58.994922 26841 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36209
I20260806 01:03:58.994956 26938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36209 every 8 connection(s)
I20260806 01:03:58.996338 26841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:03:59.002315 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 26841
I20260806 01:03:59.003297 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:37477
--local_ip_for_outbound_sockets=127.5.108.129
--tserver_master_addrs=127.5.108.190:36209
--webserver_port=40801
--webserver_interface=127.5.108.129
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:03:59.003405 26940 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:03:59.008340 26940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Bootstrap starting.
I20260806 01:03:59.028945 26940 log.cc:824] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Log is configured to *not* fsync() on all Append() calls
I20260806 01:03:59.040388 26940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:03:59.040902 26940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Bootstrap complete.
I20260806 01:03:59.049672 26940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:03:59.050487 26940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d00e5bf8204c268965b4610d020884, State: Initialized, Role: FOLLOWER
I20260806 01:03:59.051097 26940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:03:59.051338 26940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:03:59.051534 26940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:03:59.051761 26940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:03:59.055459 26940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:03:59.056013 26940 leader_election.cc:302] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d00e5bf8204c268965b4610d020884; no voters:
I20260806 01:03:59.056624 26940 leader_election.cc:288] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:03:59.056819 26944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:03:59.058104 26944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 2 LEADER]: Becoming Leader. State: Replica: 65d00e5bf8204c268965b4610d020884, State: Running, Role: LEADER
I20260806 01:03:59.058750 26940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:03:59.058952 26944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:03:59.061674 26946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d00e5bf8204c268965b4610d020884. Latest consensus state: current_term: 2 leader_uuid: "65d00e5bf8204c268965b4610d020884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } } }
I20260806 01:03:59.062477 26946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:59.062134 26945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "65d00e5bf8204c268965b4610d020884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } } }
I20260806 01:03:59.062939 26945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: This master's current role is: LEADER
I20260806 01:03:59.074844 26956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:03:59.082201 26956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:03:59.083557 26956 catalog_manager.cc:1259] Loaded cluster ID: 4d397331e5634217a8ff8dc0bce1a2ff
I20260806 01:03:59.083658 26956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:03:59.086692 26956 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 01:03:59.103276 26956 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 01:03:59.112260 26956 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:03:59.113694 26956 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Loaded TSK: 0
I20260806 01:03:59.115044 26956 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:03:59.336122 26942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:59.336359 26942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:59.336493 26942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:59.343485 26942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:59.343573 26942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:59.343719 26942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:03:59.352881 26942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:37477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=40801
--tserver_master_addrs=127.5.108.190:36209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:59.354008 26942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:59.355237 26942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:59.367714 26985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:59.367941 26986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:59.368024 26988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:59.368515 26942 server_base.cc:1034] running on GCE node
I20260806 01:03:59.369035 26942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:59.369968 26942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:59.371184 26942 hybrid_clock.cc:648] HybridClock initialized: now 1785978239371127 us; error 48 us; skew 500 ppm
I20260806 01:03:59.371515 26942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:59 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978239, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:59.374696 26942 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:03:59.376148 26942 webserver.cc:460] Webserver started at http://127.5.108.129:40801/ using document root <none> and password file <none>
I20260806 01:03:59.376778 26942 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:59.376911 26942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:59.382019 26942 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 01:03:59.384703 26999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:59.386052 26942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:03:59.386226 26942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "b1fed31bf91448a3b44d4b72c39e0395"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "593d310436f4d0ce9df069df24d63077"
server_key_iv: "d6efebc00652354302f5ec1783d8e311"
server_key_version: "encryptionkey@0"
I20260806 01:03:59.387118 26942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:59.414467 26942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:59.418478 26942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:59.418754 26942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:59.419709 26942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:59.420979 26942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:59.421087 26942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:59.421203 26942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:59.421306 26942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:59.453404 26942 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:37477
I20260806 01:03:59.453522 27172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:37477 every 8 connection(s)
I20260806 01:03:59.454679 26942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:03:59.462561 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 26942
I20260806 01:03:59.463418 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:40545
--local_ip_for_outbound_sockets=127.5.108.130
--tserver_master_addrs=127.5.108.190:36209
--webserver_port=37461
--webserver_interface=127.5.108.130
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:03:59 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978239, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:59.478631 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:59.457271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:42571 (local address 127.5.108.190:36209)
0806 01:03:59.457746 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:59.457757 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:59.458435 (+ 678us) server_negotiation.cc:407] Connection header received
0806 01:03:59.460808 (+ 2373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:59.460819 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:59.461123 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:59.461402 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:59.462757 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:59.464627 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:59.466078 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:59.467059 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:59.467243 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:59.470664 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:59.470675 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:59.470693 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:59.470751 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:59.474575 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:59.475461 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:59.475467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:59.475471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:59.475636 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:59.476143 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:59.476146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:59.476148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:59.476559 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:59.476705 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:59.477502 (+ 797us) server_negotiation.cc:299] Negotiation successful
0806 01:03:59.477809 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":205,"threads_started":1}
I20260806 01:03:59.479118 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:59.457484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:42571)
0806 01:03:59.458285 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:59.458325 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:59.460202 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:59.461526 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:59.461532 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:59.462000 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:59.462477 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:59.462492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:59.464786 (+ 2294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:59.464790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:59.465928 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:59.465950 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:59.467254 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:59.467258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:59.467483 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:59.468062 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:59.468081 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:59.470359 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:59.474699 (+ 4340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:59.474704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:59.474714 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:59.475290 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:59.475766 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:59.475772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:59.475777 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:59.476019 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:59.476716 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:59.476726 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:59.477080 (+ 354us) client_negotiation.cc:769] Sending connection context
0806 01:03:59.477345 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 01:03:59.477841 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":350,"threads_started":1}
I20260806 01:03:59.481877 27173 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36209
I20260806 01:03:59.482158 27173 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:59.482797 27173 heartbeater.cc:511] Master 127.5.108.190:36209 requested a full tablet report, sending...
I20260806 01:03:59.485219 26886 ts_manager.cc:194] Registered new tserver with Master: b1fed31bf91448a3b44d4b72c39e0395 (127.5.108.129:37477)
I20260806 01:03:59.487406 26886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:42571
W20260806 01:03:59.793079 27179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:03:59.793305 27179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:03:59.793386 27179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:03:59.799938 27179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:03:59.800029 27179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:03:59.800132 27179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:03:59.808787 27179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:40545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=37461
--tserver_master_addrs=127.5.108.190:36209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:03:59.809865 27179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:03:59.811048 27179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:03:59.823486 27187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:59.823458 27188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:03:59.823521 27190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:03:59.824489 27179 server_base.cc:1034] running on GCE node
I20260806 01:03:59.825039 27179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:03:59.825889 27179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:03:59.827118 27179 hybrid_clock.cc:648] HybridClock initialized: now 1785978239827075 us; error 54 us; skew 500 ppm
I20260806 01:03:59.827469 27179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:03:59 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978239, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:03:59.830451 27179 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:03:59.831903 27179 webserver.cc:460] Webserver started at http://127.5.108.130:37461/ using document root <none> and password file <none>
I20260806 01:03:59.832571 27179 fs_manager.cc:359] Metadata directory not provided
I20260806 01:03:59.832741 27179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:03:59.837652 27179 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:03:59.840304 27201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:59.841506 27179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:03:59.841687 27179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "4c3251d466e04c2eb532c2e4841e017f"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "0ef9acc72df4cbae7c31a8a0224c7216"
server_key_iv: "51bd7d0e0df1b5d32c0d1d88aa1a02b4"
server_key_version: "encryptionkey@0"
I20260806 01:03:59.842711 27179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:03:59.867916 27179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:03:59.872628 27179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:03:59.872910 27179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:03:59.873919 27179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:03:59.875123 27179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:03:59.875267 27179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:03:59.875406 27179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:03:59.875517 27179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 01:03:59.908314 27179 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:40545
I20260806 01:03:59.908396 27378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:40545 every 8 connection(s)
I20260806 01:03:59.909698 27179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:03:59.912684 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 27179
I20260806 01:03:59.913268 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:33523
--local_ip_for_outbound_sockets=127.5.108.131
--tserver_master_addrs=127.5.108.190:36209
--webserver_port=34253
--webserver_interface=127.5.108.131
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:03:59 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978239, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:03:59.929658 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:59.912213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:37101 (local address 127.5.108.190:36209)
0806 01:03:59.912431 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:03:59.912438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:03:59.913711 (+ 1273us) server_negotiation.cc:407] Connection header received
0806 01:03:59.915569 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:03:59.915577 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:03:59.915672 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:03:59.915788 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:03:59.917127 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:59.918236 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:59.919717 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:03:59.920449 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:03:59.920607 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:59.923959 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:03:59.923967 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:03:59.923975 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:03:59.924018 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:03:59.926499 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:59.927443 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:59.927447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:59.927449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:59.927564 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:03:59.928057 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:03:59.928060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:03:59.928062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:03:59.928297 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:03:59.928403 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 01:03:59.929225 (+ 822us) server_negotiation.cc:299] Negotiation successful
0806 01:03:59.929410 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:03:59.931035 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:03:59.912714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:37101)
0806 01:03:59.913536 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0806 01:03:59.913577 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:03:59.915243 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:03:59.915971 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:03:59.915978 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:03:59.916425 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:03:59.916944 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:59.916959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:59.918386 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:59.918389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:59.919551 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:03:59.919564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:03:59.920672 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:03:59.920677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:03:59.920850 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:03:59.921483 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:03:59.921503 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:03:59.923789 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:03:59.926632 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:59.926642 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:59.926658 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:59.927287 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:59.927670 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:03:59.927677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:03:59.927683 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:03:59.927939 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:03:59.928433 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:03:59.928443 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:03:59.928826 (+ 383us) client_negotiation.cc:769] Sending connection context
0806 01:03:59.929119 (+ 293us) client_negotiation.cc:240] Negotiation successful
0806 01:03:59.929716 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":349,"threads_started":1}
I20260806 01:03:59.932354 27379 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36209
I20260806 01:03:59.932649 27379 heartbeater.cc:464] Registering TS with master...
I20260806 01:03:59.933285 27379 heartbeater.cc:511] Master 127.5.108.190:36209 requested a full tablet report, sending...
I20260806 01:03:59.934581 26886 ts_manager.cc:194] Registered new tserver with Master: 4c3251d466e04c2eb532c2e4841e017f (127.5.108.130:40545)
I20260806 01:03:59.935357 26886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:37101
W20260806 01:04:00.236299 27383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:00.236590 27383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:00.236750 27383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:00.243336 27383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:00.243490 27383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:00.243659 27383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:04:00.252847 27383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:33523
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=34253
--tserver_master_addrs=127.5.108.190:36209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:00.253962 27383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:00.255194 27383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:00.267558 27391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:00.267521 27393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:00.267652 27396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:00.268138 27383 server_base.cc:1034] running on GCE node
I20260806 01:04:00.268698 27383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:00.269594 27383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:00.270794 27383 hybrid_clock.cc:648] HybridClock initialized: now 1785978240270741 us; error 54 us; skew 500 ppm
I20260806 01:04:00.271087 27383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:00 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978240, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:00.274220 27383 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:04:00.275594 27383 webserver.cc:460] Webserver started at http://127.5.108.131:34253/ using document root <none> and password file <none>
I20260806 01:04:00.276208 27383 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:00.276355 27383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:00.281070 27383 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 01:04:00.283480 27407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:00.284749 27383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:04:00.284935 27383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "cd734bb0b7df4d599266b27199dce349"
format_stamp: "Formatted at 2026-08-06 01:03:57 on dist-test-slave-x4qh"
server_key: "3fbdfd36b5ae65fe6dd47d99c5052b31"
server_key_iv: "a6b0bbdeeb3ad67a59cc2343d446593c"
server_key_version: "encryptionkey@0"
I20260806 01:04:00.285853 27383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:00.306916 27383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:00.310950 27383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:00.311231 27383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:00.312146 27383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:00.313375 27383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:00.313494 27383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:00.313625 27383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:00.313740 27383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:00.346256 27383 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:33523
I20260806 01:04:00.346350 27584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:33523 every 8 connection(s)
I20260806 01:04:00.347935 27383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:04:00.353408 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 27383
Aug 06 01:04:00 dist-test-slave-x4qh krb5kdc[26006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978240, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:04:00.368945 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.354251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.5.108.190:36209)
0806 01:04:00.354775 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.354780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.355081 (+ 301us) server_negotiation.cc:407] Connection header received
0806 01:04:00.355252 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.355255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.355304 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.355439 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:00.357049 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.357699 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.359513 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.360190 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.360312 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.363011 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:00.363017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:00.363022 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:00.363060 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:00.366265 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.366979 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.366991 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.366994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.367125 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.367563 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.367579 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.367582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.367839 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:00.368074 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.368364 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.368560 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":271,"threads_started":1}
I20260806 01:04:00.369226 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.350448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:40209 (local address 127.5.108.190:36209)
0806 01:04:00.350711 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.350717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.351863 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 01:04:00.353678 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.353684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.353769 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.353884 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:00.355870 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.356531 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.358241 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.359082 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.359210 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.363142 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:00.363147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:00.363150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:00.363188 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:00.366421 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.367111 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.367115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.367117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.367230 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.367728 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.367732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.367734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.367959 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:00.368097 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.368669 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.368854 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 01:04:00.370173 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.350963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:40209)
0806 01:04:00.351699 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:00.351750 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:00.353378 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:00.354174 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:00.354185 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:00.354899 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:04:00.355687 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.355712 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.356681 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.356686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.358109 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.358123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.359334 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.359343 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.359772 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.360649 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:04:00.360676 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:04:00.362906 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:04:00.366554 (+ 3648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:00.366563 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:00.366582 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:00.366973 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:00.367352 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:00.367357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:00.367362 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:00.367595 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:00.368080 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:04:00.368087 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:04:00.368356 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 01:04:00.368576 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 01:04:00.369024 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":277,"threads_started":1}
I20260806 01:04:00.371513 27585 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36209
I20260806 01:04:00.371798 27585 heartbeater.cc:464] Registering TS with master...
I20260806 01:04:00.372416 27585 heartbeater.cc:511] Master 127.5.108.190:36209 requested a full tablet report, sending...
I20260806 01:04:00.373617 26886 ts_manager.cc:194] Registered new tserver with Master: cd734bb0b7df4d599266b27199dce349 (127.5.108.131:33523)
I20260806 01:04:00.374377 26886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:40209
I20260806 01:04:00.383920 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:04:00.390143 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.384794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.5.108.190:36209)
0806 01:04:00.384925 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.384929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.385411 (+ 482us) server_negotiation.cc:407] Connection header received
0806 01:04:00.385563 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.385566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.385682 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.385806 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:04:00.386170 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.386820 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.388193 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.388621 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.388747 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.389180 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:04:00.389628 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:04:00.389730 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.389849 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.389931 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 01:04:00.390990 26886 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:04:00.397584 26886 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:04:00.428275 27605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.410168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.5.108.129:37477)
0806 01:04:00.410896 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.410908 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.417063 (+ 6155us) server_negotiation.cc:407] Connection header received
0806 01:04:00.417515 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.417526 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.417840 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.418023 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:00.419106 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.421077 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.425288 (+ 4211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.426771 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.426902 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.427085 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.427797 (+ 712us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.427956 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":346,"threads_started":1}
I20260806 01:04:00.428318 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.408612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37477 (local address 127.0.0.1:39258)
0806 01:04:00.416871 (+ 8259us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:00.416921 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:00.417230 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:00.418387 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:00.418394 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:00.418423 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:00.418844 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.418860 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.422489 (+ 3629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.422497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.425119 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.425133 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.427247 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.427259 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.427524 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.427587 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 01:04:00.427738 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 01:04:00.427901 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7118,"mutex_wait_us":85,"thread_start_us":6969,"threads_started":1}
I20260806 01:04:00.430990 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.420968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:33523 (local address 127.0.0.1:52568)
0806 01:04:00.423493 (+ 2525us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:00.423515 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:00.423630 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:00.424415 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:00.424418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:00.424441 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:00.424913 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.424927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.426378 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.426383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.428526 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.428544 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.430113 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.430119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.430323 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.430361 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:04:00.430465 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 01:04:00.430594 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2387,"mutex_wait_us":61,"thread_start_us":1841,"threads_started":1}
I20260806 01:04:00.431003 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.416662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52568 (local address 127.5.108.131:33523)
0806 01:04:00.421112 (+ 4450us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.421123 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.423616 (+ 2493us) server_negotiation.cc:407] Connection header received
0806 01:04:00.423778 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.423787 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.423989 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.424169 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:00.425140 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.426188 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.428728 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.429903 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.430057 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.430280 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.430509 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.430673 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4232,"thread_start_us":4025,"threads_started":1}
I20260806 01:04:00.431493 27608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.408964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:40545 (local address 127.0.0.1:49264)
0806 01:04:00.423173 (+ 14209us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:00.423191 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:00.423323 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:00.424086 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:00.424091 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:00.424142 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:00.424525 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.424542 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.426479 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.426486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.428736 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.428757 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.430687 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.430693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.430839 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.430871 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:04:00.430962 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 01:04:00.431072 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14048,"mutex_wait_us":23,"thread_start_us":14131,"threads_started":1}
I20260806 01:04:00.432205 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.409651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.5.108.130:40545)
0806 01:04:00.422446 (+ 12795us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.422457 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.423310 (+ 853us) server_negotiation.cc:407] Connection header received
0806 01:04:00.423504 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.423510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.423744 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.423931 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:00.424724 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.426302 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.428877 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.430107 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.430262 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.430506 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.431738 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.431895 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12606,"thread_start_us":12803,"threads_started":1}
I20260806 01:04:00.432425 27070 tablet_service.cc:1467] Processing CreateTablet for tablet f98f741d354d4334856d152da08bc26f (DEFAULT_TABLE table=test-table [id=2a10e798dd5b46c190cded62b7b76634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:04:00.435475 27278 tablet_service.cc:1467] Processing CreateTablet for tablet f98f741d354d4334856d152da08bc26f (DEFAULT_TABLE table=test-table [id=2a10e798dd5b46c190cded62b7b76634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:04:00.436064 27070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98f741d354d4334856d152da08bc26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:00.436671 27278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98f741d354d4334856d152da08bc26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:00.437278 27490 tablet_service.cc:1467] Processing CreateTablet for tablet f98f741d354d4334856d152da08bc26f (DEFAULT_TABLE table=test-table [id=2a10e798dd5b46c190cded62b7b76634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:04:00.438512 27490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98f741d354d4334856d152da08bc26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:00.447278 27609 tablet_bootstrap.cc:492] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: Bootstrap starting.
I20260806 01:04:00.450945 27610 tablet_bootstrap.cc:492] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: Bootstrap starting.
I20260806 01:04:00.451176 27609 tablet_bootstrap.cc:654] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:00.452476 27609 log.cc:824] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:00.452561 27612 tablet_bootstrap.cc:492] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: Bootstrap starting.
I20260806 01:04:00.455125 27612 tablet_bootstrap.cc:654] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:00.456362 27612 log.cc:824] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:00.457724 27610 tablet_bootstrap.cc:654] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:00.459211 27610 log.cc:824] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:00.462409 27609 tablet_bootstrap.cc:492] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: No bootstrap required, opened a new log
I20260806 01:04:00.462698 27609 ts_tablet_manager.cc:1397] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260806 01:04:00.465875 27612 tablet_bootstrap.cc:492] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: No bootstrap required, opened a new log
I20260806 01:04:00.466184 27612 ts_tablet_manager.cc:1397] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.013s
I20260806 01:04:00.466239 27610 tablet_bootstrap.cc:492] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: No bootstrap required, opened a new log
I20260806 01:04:00.466481 27610 ts_tablet_manager.cc:1397] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.011s
I20260806 01:04:00.472457 27609 raft_consensus.cc:348] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } }
I20260806 01:04:00.472828 27609 raft_consensus.cc:374] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:00.472950 27609 raft_consensus.cc:729] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1fed31bf91448a3b44d4b72c39e0395, State: Initialized, Role: FOLLOWER
I20260806 01:04:00.473556 27609 consensus_queue.cc:260] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } }
I20260806 01:04:00.474677 27173 heartbeater.cc:503] Master 127.5.108.190:36209 was elected leader, sending a full tablet report...
I20260806 01:04:00.475327 27609 ts_tablet_manager.cc:1428] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: Time spent starting tablet: real 0.012s user 0.008s sys 0.003s
I20260806 01:04:00.476761 27610 raft_consensus.cc:348] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } }
I20260806 01:04:00.476755 27612 raft_consensus.cc:348] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } }
I20260806 01:04:00.477169 27610 raft_consensus.cc:374] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:00.477186 27612 raft_consensus.cc:374] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:00.477291 27612 raft_consensus.cc:729] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd734bb0b7df4d599266b27199dce349, State: Initialized, Role: FOLLOWER
I20260806 01:04:00.477291 27610 raft_consensus.cc:729] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c3251d466e04c2eb532c2e4841e017f, State: Initialized, Role: FOLLOWER
I20260806 01:04:00.477890 27612 consensus_queue.cc:260] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } }
I20260806 01:04:00.477890 27610 consensus_queue.cc:260] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } }
I20260806 01:04:00.478969 27379 heartbeater.cc:503] Master 127.5.108.190:36209 was elected leader, sending a full tablet report...
I20260806 01:04:00.479853 27610 ts_tablet_manager.cc:1428] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260806 01:04:00.480772 27585 heartbeater.cc:503] Master 127.5.108.190:36209 was elected leader, sending a full tablet report...
I20260806 01:04:00.481392 27612 ts_tablet_manager.cc:1428] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260806 01:04:00.495335 27621 raft_consensus.cc:482] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:04:00.495594 27621 raft_consensus.cc:504] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } }
I20260806 01:04:00.496933 27621 leader_election.cc:288] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1fed31bf91448a3b44d4b72c39e0395 (127.5.108.129:37477), 4c3251d466e04c2eb532c2e4841e017f (127.5.108.130:40545)
I20260806 01:04:00.502100 27605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.497233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:34835 (local address 127.5.108.129:37477)
0806 01:04:00.497401 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.497408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.497432 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:04:00.497592 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.497595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.497644 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.497745 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:00.498113 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.498774 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.500437 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.501160 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.501330 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.501408 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.501718 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.501853 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:04:00.502159 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.497132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37477 (local address 127.5.108.131:34835)
0806 01:04:00.497305 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:00.497322 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:00.497426 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:00.497769 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:00.497774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:00.497802 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:00.497976 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.498005 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.498899 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.498904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.500281 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.500293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.501320 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.501326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.501532 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.501554 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 01:04:00.501691 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 01:04:00.501852 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":1}
I20260806 01:04:00.502835 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.497329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:57661 (local address 127.5.108.130:40545)
0806 01:04:00.497496 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.497501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.498026 (+ 525us) server_negotiation.cc:407] Connection header received
0806 01:04:00.498173 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.498176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.498222 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.498316 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:00.498781 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.499455 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.501118 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.501871 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.502044 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.502111 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.502493 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.502614 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:04:00.502939 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.497138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:40545 (local address 127.5.108.131:57661)
0806 01:04:00.497930 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:00.497943 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:00.498038 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:00.498331 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:00.498335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:00.498357 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:00.498643 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.498650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.499607 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.499611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.500974 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:00.500984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.502067 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.502073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:00.502298 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.502327 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 01:04:00.502447 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 01:04:00.502565 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":176,"thread_start_us":322,"threads_started":1}
I20260806 01:04:00.503160 27100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98f741d354d4334856d152da08bc26f" candidate_uuid: "cd734bb0b7df4d599266b27199dce349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1fed31bf91448a3b44d4b72c39e0395" is_pre_election: true
I20260806 01:04:00.503695 27100 raft_consensus.cc:2454] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd734bb0b7df4d599266b27199dce349 in term 0.
I20260806 01:04:00.503947 27312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98f741d354d4334856d152da08bc26f" candidate_uuid: "cd734bb0b7df4d599266b27199dce349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c3251d466e04c2eb532c2e4841e017f" is_pre_election: true
I20260806 01:04:00.504424 27409 leader_election.cc:302] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1fed31bf91448a3b44d4b72c39e0395, cd734bb0b7df4d599266b27199dce349; no voters:
I20260806 01:04:00.504456 27312 raft_consensus.cc:2454] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd734bb0b7df4d599266b27199dce349 in term 0.
I20260806 01:04:00.504837 27621 raft_consensus.cc:2781] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:04:00.505020 27621 raft_consensus.cc:482] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:04:00.505154 27621 raft_consensus.cc:3037] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:00.507699 27621 raft_consensus.cc:504] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } }
I20260806 01:04:00.508221 27621 leader_election.cc:288] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [CANDIDATE]: Term 1 election: Requested vote from peers b1fed31bf91448a3b44d4b72c39e0395 (127.5.108.129:37477), 4c3251d466e04c2eb532c2e4841e017f (127.5.108.130:40545)
I20260806 01:04:00.508622 27100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98f741d354d4334856d152da08bc26f" candidate_uuid: "cd734bb0b7df4d599266b27199dce349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1fed31bf91448a3b44d4b72c39e0395"
I20260806 01:04:00.508687 27312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98f741d354d4334856d152da08bc26f" candidate_uuid: "cd734bb0b7df4d599266b27199dce349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c3251d466e04c2eb532c2e4841e017f"
I20260806 01:04:00.508805 27100 raft_consensus.cc:3037] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:00.508877 27312 raft_consensus.cc:3037] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:00.510965 27312 raft_consensus.cc:2454] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd734bb0b7df4d599266b27199dce349 in term 1.
I20260806 01:04:00.511013 27100 raft_consensus.cc:2454] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd734bb0b7df4d599266b27199dce349 in term 1.
I20260806 01:04:00.511360 27411 leader_election.cc:302] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c3251d466e04c2eb532c2e4841e017f, cd734bb0b7df4d599266b27199dce349; no voters:
I20260806 01:04:00.511687 27621 raft_consensus.cc:2781] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:04:00.512054 27621 raft_consensus.cc:686] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 1 LEADER]: Becoming Leader. State: Replica: cd734bb0b7df4d599266b27199dce349, State: Running, Role: LEADER
I20260806 01:04:00.512434 27621 consensus_queue.cc:237] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } }
I20260806 01:04:00.516439 26884 catalog_manager.cc:5495] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd734bb0b7df4d599266b27199dce349 (127.5.108.131). New cstate: current_term: 1 leader_uuid: "cd734bb0b7df4d599266b27199dce349" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd734bb0b7df4d599266b27199dce349" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33523 } health_report { overall_health: HEALTHY } } }
I20260806 01:04:00.541953 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.536656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52582 (local address 127.5.108.131:33523)
0806 01:04:00.536863 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.536869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.536970 (+ 101us) server_negotiation.cc:407] Connection header received
0806 01:04:00.537191 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.537195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.537263 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.537400 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:04:00.537782 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.538468 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.540075 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.540470 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.540613 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.541059 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:04:00.541440 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:04:00.541586 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.541667 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.541733 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:04:00.549055 27312 raft_consensus.cc:1261] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [term 1 FOLLOWER]: Refusing update from remote peer cd734bb0b7df4d599266b27199dce349: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:04:00.549053 27100 raft_consensus.cc:1261] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [term 1 FOLLOWER]: Refusing update from remote peer cd734bb0b7df4d599266b27199dce349: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:04:00.550001 27623 consensus_queue.cc:1035] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1fed31bf91448a3b44d4b72c39e0395" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:04:00.550315 27621 consensus_queue.cc:1035] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c3251d466e04c2eb532c2e4841e017f" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:04:00.573235 26886 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:04:00.573468 26886 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:04:00.576678 26886 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Sending DeleteTablet for 3 replicas of tablet f98f741d354d4334856d152da08bc26f
I20260806 01:04:00.577723 27278 tablet_service.cc:1514] Processing DeleteTablet for tablet f98f741d354d4334856d152da08bc26f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:04:00 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:49264
I20260806 01:04:00.577811 27070 tablet_service.cc:1514] Processing DeleteTablet for tablet f98f741d354d4334856d152da08bc26f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:04:00 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:39258
I20260806 01:04:00.578078 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 26942
I20260806 01:04:00.578719 27648 tablet_replica.cc:330] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: stopping tablet replica
I20260806 01:04:00.579087 27490 tablet_service.cc:1514] Processing DeleteTablet for tablet f98f741d354d4334856d152da08bc26f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:04:00 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:52568
I20260806 01:04:00.579245 27650 tablet_replica.cc:330] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: stopping tablet replica
I20260806 01:04:00.579180 27648 raft_consensus.cc:2229] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:04:00.579681 27650 raft_consensus.cc:2229] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:04:00.579983 27648 raft_consensus.cc:2258] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:04:00.580156 27650 raft_consensus.cc:2258] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:04:00.582268 27650 ts_tablet_manager.cc:1905] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:04:00.587159 27648 ts_tablet_manager.cc:1905] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:04:00.587926 27651 tablet_replica.cc:330] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: stopping tablet replica
I20260806 01:04:00.588299 27651 raft_consensus.cc:2229] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:04:00.588979 27651 raft_consensus.cc:2258] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:04:00.591255 27650 ts_tablet_manager.cc:1918] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:04:00.591441 27650 log.cc:1196] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/f98f741d354d4334856d152da08bc26f
I20260806 01:04:00.591967 27650 ts_tablet_manager.cc:1939] T f98f741d354d4334856d152da08bc26f P b1fed31bf91448a3b44d4b72c39e0395: Deleting consensus metadata
I20260806 01:04:00.593565 26867 catalog_manager.cc:4841] TS b1fed31bf91448a3b44d4b72c39e0395 (127.5.108.129:37477): tablet f98f741d354d4334856d152da08bc26f (table test-table [id=2a10e798dd5b46c190cded62b7b76634]) successfully deleted
I20260806 01:04:00.594795 27651 ts_tablet_manager.cc:1905] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:04:00.597591 27605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.579659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.5.108.129:37477)
0806 01:04:00.579843 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.579851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.579876 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:04:00.579931 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.579936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.580008 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.580104 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:00.582180 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.583779 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.585612 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.586462 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.586631 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.590495 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:00.590510 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:00.590528 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:00.590591 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:00.594386 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.595315 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.595324 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.595328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.595515 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.595939 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.595944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.595947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.596415 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:00.596712 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.597034 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.597247 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":64}
I20260806 01:04:00.598272 27166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:00.600312 27648 ts_tablet_manager.cc:1918] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:04:00.600476 27648 log.cc:1196] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/f98f741d354d4334856d152da08bc26f
I20260806 01:04:00.601018 27648 ts_tablet_manager.cc:1939] T f98f741d354d4334856d152da08bc26f P 4c3251d466e04c2eb532c2e4841e017f: Deleting consensus metadata
I20260806 01:04:00.602736 26871 catalog_manager.cc:4841] TS 4c3251d466e04c2eb532c2e4841e017f (127.5.108.130:40545): tablet f98f741d354d4334856d152da08bc26f (table test-table [id=2a10e798dd5b46c190cded62b7b76634]) successfully deleted
I20260806 01:04:00.606994 27651 ts_tablet_manager.cc:1918] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:04:00.607156 27651 log.cc:1196] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/f98f741d354d4334856d152da08bc26f
I20260806 01:04:00.607729 27651 ts_tablet_manager.cc:1939] T f98f741d354d4334856d152da08bc26f P cd734bb0b7df4d599266b27199dce349: Deleting consensus metadata
I20260806 01:04:00.609221 26867 catalog_manager.cc:4841] TS cd734bb0b7df4d599266b27199dce349 (127.5.108.131:33523): tablet f98f741d354d4334856d152da08bc26f (table test-table [id=2a10e798dd5b46c190cded62b7b76634]) successfully deleted
I20260806 01:04:00.688340 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 26942
I20260806 01:04:00.713376 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 27179
I20260806 01:04:00.728323 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.714222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.5.108.130:40545)
0806 01:04:00.714448 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.714454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.714477 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:04:00.714580 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.714587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.714673 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.714781 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:00.716124 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.717233 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.718494 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.719121 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.719251 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.721911 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:00.721928 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:00.721947 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:00.722002 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:00.725637 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.726262 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.726270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.726273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.726415 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.726724 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.726728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.726731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.727168 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:00.727506 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.727732 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.727961 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:04:00.728986 27372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:00.814863 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 27179
I20260806 01:04:00.844894 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 27383
I20260806 01:04:00.859618 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:00.845758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52590 (local address 127.5.108.131:33523)
0806 01:04:00.845940 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:00.845946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:00.845964 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:04:00.846159 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:00.846164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:00.846250 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:00.846389 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:00.847935 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.848814 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.850218 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:00.850709 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:00.850852 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:00.853459 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:00.853485 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:00.853500 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:00.853551 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:00.856523 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.857157 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.857169 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.857172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.857356 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:00.857760 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:00.857765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:00.857769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:00.858111 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:00.858482 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:00.858805 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 01:04:00.859082 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:04:00.860482 27572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:00.978809 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 27383
I20260806 01:04:01.008463 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 26841
I20260806 01:04:01.009644 26932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:01.116880 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 26841
I20260806 01:04:01.146339 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36209
--webserver_interface=127.5.108.190
--webserver_port=36059
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:04:01.466454 27657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:01.466790 27657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:01.466952 27657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:01.474438 27657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:04:01.474599 27657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:01.474697 27657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:01.474814 27657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:04:01.474902 27657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:04:01.484019 27657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=36059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:01.485199 27657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:01.486435 27657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:01.497504 27664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:01.497545 27668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:01.498581 27666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:01.498817 27657 server_base.cc:1034] running on GCE node
I20260806 01:04:01.499671 27657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:01.500952 27657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:01.502182 27657 hybrid_clock.cc:648] HybridClock initialized: now 1785978241502143 us; error 57 us; skew 500 ppm
I20260806 01:04:01.502504 27657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:01 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978241, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:01.505640 27657 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:04:01.510409 27657 webserver.cc:460] Webserver started at http://127.5.108.190:36059/ using document root <none> and password file <none>
I20260806 01:04:01.511313 27657 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:01.511507 27657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:01.517313 27657 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260806 01:04:01.520071 27677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:01.521380 27657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:04:01.521554 27657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "65d00e5bf8204c268965b4610d020884"
format_stamp: "Formatted at 2026-08-06 01:03:55 on dist-test-slave-x4qh"
server_key: "9c4d5689c3510e3e211880dc0337369f"
server_key_iv: "71580501e35ab79b27a1bed4cbfc28c2"
server_key_version: "encryptionkey@0"
I20260806 01:04:01.522437 27657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:01.535663 27657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:01.539304 27657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:01.539597 27657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:01.558075 27657 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36209
I20260806 01:04:01.558108 27749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36209 every 8 connection(s)
I20260806 01:04:01.559546 27657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:04:01.561683 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 27657
I20260806 01:04:01.562716 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:37477
--local_ip_for_outbound_sockets=127.5.108.129
--tserver_master_addrs=127.5.108.190:36209
--webserver_port=40801
--webserver_interface=127.5.108.129
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:04:01.568759 27752 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:04:01.573724 27752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Bootstrap starting.
I20260806 01:04:01.594969 27752 log.cc:824] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:01.610688 27752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:04:01.611227 27752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884: Bootstrap complete.
I20260806 01:04:01.621110 27752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:04:01.621949 27752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d00e5bf8204c268965b4610d020884, State: Initialized, Role: FOLLOWER
I20260806 01:04:01.622572 27752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:04:01.622802 27752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:04:01.622927 27752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:04:01.623116 27752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 2 FOLLOWER]: Advancing to term 3
I20260806 01:04:01.626981 27752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:04:01.627503 27752 leader_election.cc:302] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d00e5bf8204c268965b4610d020884; no voters:
I20260806 01:04:01.628229 27752 leader_election.cc:288] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 01:04:01.628361 27759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 01:04:01.630705 27752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:04:01.635582 27759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [term 3 LEADER]: Becoming Leader. State: Replica: 65d00e5bf8204c268965b4610d020884, State: Running, Role: LEADER
I20260806 01:04:01.636216 27759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } }
I20260806 01:04:01.639381 27762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d00e5bf8204c268965b4610d020884. Latest consensus state: current_term: 3 leader_uuid: "65d00e5bf8204c268965b4610d020884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } } }
I20260806 01:04:01.640021 27762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: This master's current role is: LEADER
I20260806 01:04:01.638959 27761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "65d00e5bf8204c268965b4610d020884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d00e5bf8204c268965b4610d020884" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36209 } } }
I20260806 01:04:01.640720 27761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d00e5bf8204c268965b4610d020884 [sys.catalog]: This master's current role is: LEADER
I20260806 01:04:01.648972 27786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:04:01.656870 27786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:04:01.658257 27786 catalog_manager.cc:1259] Loaded cluster ID: 4d397331e5634217a8ff8dc0bce1a2ff
I20260806 01:04:01.658344 27786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 01:04:01.660315 27786 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0006fc29ad google::LogMessage::Fail() at ??:0
@ 0x7f0006fc588a google::LogMessage::SendToLog() at ??:0
@ 0x7f0006fc24de google::LogMessage::Flush() at ??:0
@ 0x7f0006fc3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f001c9f5512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f001c9f2976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f001ca7a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f001ca7a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0009c0e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0009c1191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0009c11799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0009beceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f000cec66db start_thread at ??:0
@ 0x7f0005e5c61f clone at ??:0
W20260806 01:04:01.966137 27756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:01.966377 27756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:01.966470 27756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:01.973022 27756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:01.973117 27756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:01.973222 27756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:04:01.982326 27756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:37477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=40801
--tserver_master_addrs=127.5.108.190:36209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:01.983541 27756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:01.985136 27756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:01.998101 27830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:01.998167 27827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:01.998201 27828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:01.999017 27756 server_base.cc:1034] running on GCE node
I20260806 01:04:01.999687 27756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:02.000545 27756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:02.001767 27756 hybrid_clock.cc:648] HybridClock initialized: now 1785978242001731 us; error 53 us; skew 500 ppm
I20260806 01:04:02.002090 27756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:01 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978241, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:02.005030 27756 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:04:02.006325 27756 webserver.cc:460] Webserver started at http://127.5.108.129:40801/ using document root <none> and password file <none>
I20260806 01:04:02.006963 27756 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:02.007100 27756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:02.012207 27756 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:04:02.014866 27843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:02.016125 27756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:04:02.016312 27756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "b1fed31bf91448a3b44d4b72c39e0395"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "593d310436f4d0ce9df069df24d63077"
server_key_iv: "d6efebc00652354302f5ec1783d8e311"
server_key_version: "encryptionkey@0"
I20260806 01:04:02.017252 27756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:02.036363 27756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:02.040166 27756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:02.040433 27756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:02.041348 27756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:02.042598 27756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:02.042762 27756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:02.042886 27756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:02.042981 27756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:02.076567 27756 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:37477
I20260806 01:04:02.076651 28010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:37477 every 8 connection(s)
I20260806 01:04:02.078028 27756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:04:02.082471 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:02.080809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:41065)
0806 01:04:02.081508 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:02.081734 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":289,"threads_started":1}
W20260806 01:04:02.083103 28012 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:36209 (0 consecutive failures): Network error: Failed to ping master at 127.5.108.190:36209: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
I20260806 01:04:02.083870 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:02.083586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:33511)
0806 01:04:02.083710 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:02.083773 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 01:04:02.084666 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:02.084385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:34577)
0806 01:04:02.084497 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:02.084565 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 01:04:02.084905 28012 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:04:02.086253 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 27756
I20260806 01:04:02.086817 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:40545
--local_ip_for_outbound_sockets=127.5.108.130
--tserver_master_addrs=127.5.108.190:36209
--webserver_port=37461
--webserver_interface=127.5.108.130
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:04:02.412530 28015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:02.412823 28015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:02.412984 28015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:02.419677 28015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:02.419829 28015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:02.420002 28015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:04:02.428987 28015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:40545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=37461
--tserver_master_addrs=127.5.108.190:36209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:02.430166 28015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:02.431437 28015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:02.444782 28020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:02.444862 28022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:02.444900 28024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:02.445719 28015 server_base.cc:1034] running on GCE node
I20260806 01:04:02.446369 28015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:02.447194 28015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:02.448424 28015 hybrid_clock.cc:648] HybridClock initialized: now 1785978242448371 us; error 37 us; skew 500 ppm
I20260806 01:04:02.448710 28015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:02 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978242, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:02.451833 28015 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:04:02.453156 28015 webserver.cc:460] Webserver started at http://127.5.108.130:37461/ using document root <none> and password file <none>
I20260806 01:04:02.453755 28015 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:02.453848 28015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:02.458566 28015 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:04:02.461122 28033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:02.462333 28015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:04:02.462507 28015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "4c3251d466e04c2eb532c2e4841e017f"
format_stamp: "Formatted at 2026-08-06 01:03:56 on dist-test-slave-x4qh"
server_key: "0ef9acc72df4cbae7c31a8a0224c7216"
server_key_iv: "51bd7d0e0df1b5d32c0d1d88aa1a02b4"
server_key_version: "encryptionkey@0"
I20260806 01:04:02.463419 28015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:02.487532 28015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:02.491068 28015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:02.491271 28015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:02.492065 28015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:02.493182 28015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:02.493284 28015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:02.493351 28015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:02.493407 28015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:02.526499 28015 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:40545
I20260806 01:04:02.526592 28198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:40545 every 8 connection(s)
I20260806 01:04:02.527928 28015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:04:02.532445 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:02.530738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:52551)
0806 01:04:02.531454 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:02.531710 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":248,"threads_started":1}
W20260806 01:04:02.533144 28199 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:36209 (0 consecutive failures): Network error: Failed to ping master at 127.5.108.190:36209: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
I20260806 01:04:02.533777 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 28015
I20260806 01:04:02.533979 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:02.533585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:54651)
0806 01:04:02.533770 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:02.533853 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 01:04:02.534569 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:33523
--local_ip_for_outbound_sockets=127.5.108.131
--tserver_master_addrs=127.5.108.190:36209
--webserver_port=34253
--webserver_interface=127.5.108.131
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:04:02.534897 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:02.534485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:46369)
0806 01:04:02.534614 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:02.534729 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:04:02.535166 28199 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:04:02.859213 28202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:02.859475 28202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:02.859570 28202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:02.866418 28202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:02.866506 28202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:02.866607 28202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:04:02.875424 28202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:33523
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=34253
--tserver_master_addrs=127.5.108.190:36209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:02.876523 28202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:02.877661 28202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:02.890615 28207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:02.890744 28215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:02.890807 28212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:02.891654 28202 server_base.cc:1034] running on GCE node
I20260806 01:04:02.892167 28202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:02.892995 28202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:02.894225 28202 hybrid_clock.cc:648] HybridClock initialized: now 1785978242894187 us; error 50 us; skew 500 ppm
I20260806 01:04:02.894629 28202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:02 dist-test-slave-x4qh krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978242, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:02.897547 28202 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:04:02.898856 28202 webserver.cc:460] Webserver started at http://127.5.108.131:34253/ using document root <none> and password file <none>
I20260806 01:04:02.899621 28202 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:02.899773 28202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:02.904738 28202 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 01:04:02.907267 28224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:02.908515 28202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:04:02.908690 28202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "cd734bb0b7df4d599266b27199dce349"
format_stamp: "Formatted at 2026-08-06 01:03:57 on dist-test-slave-x4qh"
server_key: "3fbdfd36b5ae65fe6dd47d99c5052b31"
server_key_iv: "a6b0bbdeeb3ad67a59cc2343d446593c"
server_key_version: "encryptionkey@0"
I20260806 01:04:02.909603 28202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:02.926064 28202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:02.929740 28202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:02.930006 28202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:02.930938 28202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:02.932240 28202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:02.932456 28202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:02.932601 28202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:02.932694 28202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:02.965562 28202 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:33523
I20260806 01:04:02.965638 28388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:33523 every 8 connection(s)
I20260806 01:04:02.967031 28202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:04:02.971525 28392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:02.969780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:55291)
0806 01:04:02.970453 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:02.970672 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":259,"threads_started":1}
I20260806 01:04:02.972041 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 28202
W20260806 01:04:02.972209 28390 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:36209 (0 consecutive failures): Network error: Failed to ping master at 127.5.108.190:36209: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
I20260806 01:04:02.972963 28392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:02.972688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:55653)
0806 01:04:02.972819 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:02.972875 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 01:04:02.973773 28392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:02.973476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:40681)
0806 01:04:02.973611 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:02.973673 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:04:02.974092 28390 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:04:03.086150 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:03.085408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:58511)
0806 01:04:03.085848 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:03.085936 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":228,"threads_started":1}
I20260806 01:04:03.536554 28395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:03.535825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:60331)
0806 01:04:03.536253 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:03.536333 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":212,"threads_started":1}
I20260806 01:04:03.975306 28396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:03.974569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:38549)
0806 01:04:03.974989 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:03.975081 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":226,"threads_started":1}
I20260806 01:04:04.087625 28397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:04.086947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:38939)
0806 01:04:04.087398 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:04.087475 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
I20260806 01:04:04.538156 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:04.537427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:59233)
0806 01:04:04.537908 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:04.537986 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":263,"threads_started":1}
I20260806 01:04:04.976873 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:04.976111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:57169)
0806 01:04:04.976623 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:04.976701 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":285,"threads_started":1}
I20260806 01:04:05.089112 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:05.088426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:55167)
0806 01:04:05.088871 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:05.088947 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":243,"threads_started":1}
I20260806 01:04:05.539788 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:05.539048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:41457)
0806 01:04:05.539519 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:05.539596 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":228,"threads_started":1}
I20260806 01:04:05.978395 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:05.977643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:41877)
0806 01:04:05.978146 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:05.978224 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":293,"threads_started":1}
I20260806 01:04:06.090601 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:06.089868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:37533)
0806 01:04:06.090341 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:06.090428 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":224,"threads_started":1}
I20260806 01:04:06.541400 28407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:06.540674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:33407)
0806 01:04:06.541111 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:06.541204 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":222,"threads_started":1}
I20260806 01:04:06.979936 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:06.979231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:51601)
0806 01:04:06.979671 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:06.979747 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":237,"threads_started":1}
I20260806 01:04:07.092641 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:07.091681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:49335)
0806 01:04:07.092255 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:07.092386 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":288,"threads_started":1}
I20260806 01:04:07.542971 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:07.542204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:53521)
0806 01:04:07.542714 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:07.542794 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":257,"threads_started":1}
I20260806 01:04:07.981931 28411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:07.981118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:39647)
0806 01:04:07.981688 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:07.981767 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":339,"threads_started":1}
I20260806 01:04:08.094393 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:08.093622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:41635)
0806 01:04:08.094139 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:08.094225 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":294,"threads_started":1}
I20260806 01:04:08.544445 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:08.543777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:50249)
0806 01:04:08.544187 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:08.544282 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":220,"threads_started":1}
I20260806 01:04:08.983421 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:08.982714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:43903)
0806 01:04:08.983153 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:08.983234 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":229,"threads_started":1}
I20260806 01:04:09.096029 28418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:09.095320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:48243)
0806 01:04:09.095752 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:09.095826 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":182,"threads_started":1}
I20260806 01:04:09.546087 28420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:09.545269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:60003)
0806 01:04:09.545725 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:09.545816 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":204,"threads_started":1}
I20260806 01:04:09.985035 28421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:09.984284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:40967)
0806 01:04:09.984761 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:09.984835 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":211,"threads_started":1}
I20260806 01:04:10.097563 28422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:10.096952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:40469)
0806 01:04:10.097345 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:10.097419 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":186,"threads_started":1}
I20260806 01:04:10.547676 28423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:10.546958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:57599)
0806 01:04:10.547446 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:10.547524 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":228,"threads_started":1}
I20260806 01:04:10.986625 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:10.985956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:35901)
0806 01:04:10.986394 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:10.986470 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260806 01:04:11.099287 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:11.098447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:44101)
0806 01:04:11.098912 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:11.099035 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":232,"threads_started":1}
I20260806 01:04:11.549197 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:11.548442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:44967)
0806 01:04:11.548945 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:11.549023 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":241,"threads_started":1}
I20260806 01:04:11.988221 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:11.987510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:38847)
0806 01:04:11.987980 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:11.988056 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":237,"threads_started":1}
I20260806 01:04:12.101193 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:12.100318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:39447)
0806 01:04:12.100906 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:12.101008 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":372,"threads_started":1}
I20260806 01:04:12.550933 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:12.550224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:55193)
0806 01:04:12.550688 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:12.550770 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":252,"threads_started":1}
I20260806 01:04:12.989815 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:12.989118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:59767)
0806 01:04:12.989588 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:12.989664 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":252,"threads_started":1}
I20260806 01:04:13.102882 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:13.102090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:36667)
0806 01:04:13.102573 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:13.102672 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":245,"threads_started":1}
I20260806 01:04:13.552567 28437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:13.551861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:49285)
0806 01:04:13.552300 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:13.552394 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":199,"threads_started":1}
I20260806 01:04:13.991636 28438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:13.990818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:50639)
0806 01:04:13.991292 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:13.991442 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":256,"threads_started":1}
I20260806 01:04:14.104624 28439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:14.103921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:59703)
0806 01:04:14.104396 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:14.104472 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":303,"threads_started":1}
I20260806 01:04:14.554275 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:14.553428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:55719)
0806 01:04:14.554000 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:14.554131 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":227,"threads_started":1}
I20260806 01:04:14.993222 28442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:14.992521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:54187)
0806 01:04:14.992993 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:14.993073 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":243,"threads_started":1}
I20260806 01:04:15.106132 28444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:15.105418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:35231)
0806 01:04:15.105854 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:15.105964 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":226,"threads_started":1}
I20260806 01:04:15.556010 28445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:15.555180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:43595)
0806 01:04:15.555732 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:15.555845 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":329,"threads_started":1}
I20260806 01:04:15.994882 28446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:15.994113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:41309)
0806 01:04:15.994584 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:15.994697 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":219,"threads_started":1}
I20260806 01:04:16.107573 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:16.106924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:55195)
0806 01:04:16.107281 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:16.107351 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":181,"threads_started":1}
I20260806 01:04:16.557446 28448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:16.556788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:43685)
0806 01:04:16.557198 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:16.557273 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":216,"threads_started":1}
I20260806 01:04:16.996429 28449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:16.995772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:55473)
0806 01:04:16.996195 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:16.996269 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":225,"threads_started":1}
I20260806 01:04:17.109483 28450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:17.108709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.129:35201)
0806 01:04:17.109212 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:17.109306 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":287,"threads_started":1}
I20260806 01:04:17.559260 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:17.558534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.130:42983)
0806 01:04:17.558945 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:17.559059 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":207,"threads_started":1}
I20260806 01:04:17.976198 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 27756
I20260806 01:04:17.993453 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:17.977274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47720 (local address 127.5.108.129:37477)
0806 01:04:17.978006 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:17.978019 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:17.978058 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:04:17.979670 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:17.979680 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:17.980078 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:17.980371 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:17.981778 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:17.982980 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:17.984300 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:17.984958 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:17.985097 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:17.987795 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:17.987803 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:17.987817 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:17.987852 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:17.990477 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:17.991157 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:17.991164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:17.991170 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:17.991343 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:17.991742 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:17.991745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:17.991747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:17.992230 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:17.992465 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:17.992775 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 01:04:17.993171 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":448,"threads_started":1}
I20260806 01:04:17.994791 28001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:17.999104 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:17.997572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36209 (local address 127.5.108.131:38609)
0806 01:04:17.998834 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:17.998921 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1150,"thread_start_us":1033,"threads_started":1}
I20260806 01:04:18.075278 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 27756
I20260806 01:04:18.100742 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 28015
I20260806 01:04:18.118954 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:18.101751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.5.108.130:40545)
0806 01:04:18.102337 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:18.102351 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:18.102388 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:04:18.103991 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:18.104000 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:18.104434 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:18.104710 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:18.106083 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:18.107330 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:18.108539 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:18.109207 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:18.109535 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:18.112603 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:18.112611 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:18.112625 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:18.112663 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:18.115907 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:18.116638 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:18.116645 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:18.116651 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:18.116839 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:18.117178 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:18.117206 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:18.117211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:18.117755 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:18.117982 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:18.118262 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 01:04:18.118641 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":300,"threads_started":1}
I20260806 01:04:18.120406 28193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:18.202160 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 28015
I20260806 01:04:18.225963 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 28202
I20260806 01:04:18.243207 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:18.226982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.5.108.131:33523)
0806 01:04:18.227639 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:18.227654 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:18.227700 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:04:18.229331 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:18.229341 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:18.229735 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:18.230024 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:18.231580 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:18.232787 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:18.234039 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:18.234715 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:18.234875 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:18.237391 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:18.237398 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:18.237410 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:18.237445 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:18.240379 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:18.240999 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:18.241005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:18.241011 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:18.241174 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:18.241478 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:18.241481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:18.241483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:18.241968 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:18.242228 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:18.242499 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 01:04:18.242895 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":275,"threads_started":1}
I20260806 01:04:18.244501 28382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:18.324589 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 28202
2026-08-06T01:04:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24627 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:04:18 dist-test-slave-x4qh krb5kdc[28465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:04:18 dist-test-slave-x4qh krb5kdc[28465](info): set up 2 sockets
Aug 06 01:04:18 dist-test-slave-x4qh krb5kdc[28465](info): commencing operation
krb5kdc: starting...
W20260806 01:04:19.572695 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.006s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:04:19 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:04:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:04:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:04:19.846597 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34403
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:34403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:04:20.171330 28485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:20.171656 28485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:20.171758 28485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:20.178751 28485 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 01:04:20.178843 28485 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 01:04:20.178880 28485 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 01:04:20.178932 28485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:04:20.178977 28485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:20.179044 28485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:20.179087 28485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:04:20.179133 28485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:04:20.188511 28485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:34403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:20.189630 28485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:20.190979 28485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:20.202386 28496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:20.202491 28497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:20.202924 28485 server_base.cc:1034] running on GCE node
W20260806 01:04:20.202491 28500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:20.203825 28485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:20.205063 28485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:20.206307 28485 hybrid_clock.cc:648] HybridClock initialized: now 1785978260206244 us; error 52 us; skew 500 ppm
I20260806 01:04:20.206595 28485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:20 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978260, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:20.209611 28485 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:04:20.211018 28485 webserver.cc:460] Webserver started at http://127.5.108.190:32913/ using document root <none> and password file <none>
I20260806 01:04:20.211724 28485 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:20.211860 28485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:20.212172 28485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:04:20.214985 28485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "cb4f642f13b8422f9615c795e0cee180"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "083b5f0825c7de14fc346563ef4ab720"
server_key_iv: "02bb50c17a42c423cfc466e9e55f1502"
server_key_version: "encryptionkey@0"
I20260806 01:04:20.215775 28485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "cb4f642f13b8422f9615c795e0cee180"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "083b5f0825c7de14fc346563ef4ab720"
server_key_iv: "02bb50c17a42c423cfc466e9e55f1502"
server_key_version: "encryptionkey@0"
I20260806 01:04:20.220461 28485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 01:04:20.223656 28512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:20.224819 28485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:04:20.224983 28485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "cb4f642f13b8422f9615c795e0cee180"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "083b5f0825c7de14fc346563ef4ab720"
server_key_iv: "02bb50c17a42c423cfc466e9e55f1502"
server_key_version: "encryptionkey@0"
I20260806 01:04:20.225179 28485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:20.242617 28485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:20.246136 28485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:20.246378 28485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:20.263082 28485 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:34403
I20260806 01:04:20.263099 28590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:34403 every 8 connection(s)
I20260806 01:04:20.264392 28485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:04:20.268096 28591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:20.270056 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 28485
I20260806 01:04:20.270277 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:04:20.270960 5554 external_mini_cluster.cc:1428] Setting key 221175220fedf43ed61e4f49c5609d0a
I20260806 01:04:20.280792 28591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Bootstrap starting.
I20260806 01:04:20.284255 28591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:20.285240 28591 log.cc:824] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:04:20 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:04:20.287910 28591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: No bootstrap required, opened a new log
I20260806 01:04:20.294673 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:20.273007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60032 (local address 127.5.108.190:34403)
0806 01:04:20.273831 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:20.273847 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:20.273897 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:04:20.276191 (+ 2294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:20.276218 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:20.276722 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:20.277172 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:20.278529 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:20.280496 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:20.281841 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:20.282952 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:20.283142 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:20.287285 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:20.287293 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:20.287306 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:20.287344 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:20.290459 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:20.291353 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:20.291360 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:20.291405 (+ 45us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:20.291587 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:20.291981 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:20.291986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:20.291989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:20.292532 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:20.292867 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:20.293302 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 01:04:20.293730 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":315,"threads_started":1}
I20260806 01:04:20.295578 28591 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:04:20.295933 28591 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:20.296062 28591 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb4f642f13b8422f9615c795e0cee180, State: Initialized, Role: FOLLOWER
I20260806 01:04:20.296574 28591 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:04:20.296835 28591 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:04:20.296983 28591 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:04:20.297152 28591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:20.299317 28591 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:04:20.299888 28591 leader_election.cc:302] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb4f642f13b8422f9615c795e0cee180; no voters:
I20260806 01:04:20.300551 28591 leader_election.cc:288] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:04:20.300706 28598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:04:20.301538 28598 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 1 LEADER]: Becoming Leader. State: Replica: cb4f642f13b8422f9615c795e0cee180, State: Running, Role: LEADER
I20260806 01:04:20.302122 28598 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:04:20.302654 28591 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:04:20.304695 28599 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb4f642f13b8422f9615c795e0cee180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } } }
I20260806 01:04:20.305035 28599 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: This master's current role is: LEADER
I20260806 01:04:20.305634 28601 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb4f642f13b8422f9615c795e0cee180. Latest consensus state: current_term: 1 leader_uuid: "cb4f642f13b8422f9615c795e0cee180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } } }
I20260806 01:04:20.305966 28604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:04:20.306404 28601 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: This master's current role is: LEADER
I20260806 01:04:20.312075 28604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:04:20.320238 28604 catalog_manager.cc:1347] Generated new cluster ID: c52bc34ea3fa4711977ffde653f039b1
I20260806 01:04:20.320340 28604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:04:20.348644 28604 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:04:20.350401 28604 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:04:20.368428 28604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Generated new TSK 0
I20260806 01:04:20.369266 28604 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:04:20.439683 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34403
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:04:20.768832 28631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:20.769088 28631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:20.769251 28631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:20.776350 28631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:20.776453 28631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:20.776635 28631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:04:20.785728 28631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:20.786912 28631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:20.788161 28631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:20.800915 28640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:20.800968 28645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:20.800990 28642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:20.802142 28631 server_base.cc:1034] running on GCE node
I20260806 01:04:20.802726 28631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:20.803709 28631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:20.804911 28631 hybrid_clock.cc:648] HybridClock initialized: now 1785978260804854 us; error 51 us; skew 500 ppm
I20260806 01:04:20.805207 28631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:20 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978260, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:20.808456 28631 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:04:20.809877 28631 webserver.cc:460] Webserver started at http://127.5.108.129:36645/ using document root <none> and password file <none>
I20260806 01:04:20.810532 28631 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:20.810671 28631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:20.810974 28631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:04:20.813869 28631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "06aa81588cf24f76a8529c4b8d74823a"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "d0a577c211d4ed2a6fe56d0f48110af3"
server_key_iv: "05ec15386a4fd3add5a8e2f8d3a3fc71"
server_key_version: "encryptionkey@0"
I20260806 01:04:20.814610 28631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "06aa81588cf24f76a8529c4b8d74823a"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "d0a577c211d4ed2a6fe56d0f48110af3"
server_key_iv: "05ec15386a4fd3add5a8e2f8d3a3fc71"
server_key_version: "encryptionkey@0"
I20260806 01:04:20.819446 28631 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260806 01:04:20.822739 28652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:20.823972 28631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:04:20.824136 28631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "06aa81588cf24f76a8529c4b8d74823a"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "d0a577c211d4ed2a6fe56d0f48110af3"
server_key_iv: "05ec15386a4fd3add5a8e2f8d3a3fc71"
server_key_version: "encryptionkey@0"
I20260806 01:04:20.824342 28631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:20.837831 28631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:20.841491 28631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:20.841787 28631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:20.842697 28631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:20.843991 28631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:20.844089 28631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:20.844226 28631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:20.844301 28631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:20.876134 28631 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:39047
I20260806 01:04:20.876215 28830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:39047 every 8 connection(s)
I20260806 01:04:20.877549 28631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:04:20.883890 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 28631
I20260806 01:04:20.884140 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:04:20.885063 5554 external_mini_cluster.cc:1428] Setting key fa8f5de83bfec70045cf4725623b20d9
Aug 06 01:04:20 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978260, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:04:20.898694 28834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:20.880142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:60827 (local address 127.5.108.190:34403)
0806 01:04:20.880575 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:20.880580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:20.881584 (+ 1004us) server_negotiation.cc:407] Connection header received
0806 01:04:20.883288 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:20.883292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:20.883356 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:20.883508 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:20.885655 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:20.886544 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:20.888289 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:20.889044 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:20.889216 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:20.893082 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:20.893086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:20.893089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:20.893119 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:20.895624 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:20.896494 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:20.896498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:20.896500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:20.896609 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:20.897055 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:20.897057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:20.897059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:20.897293 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:20.897396 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:20.898250 (+ 854us) server_negotiation.cc:299] Negotiation successful
0806 01:04:20.898425 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":221,"threads_started":1}
I20260806 01:04:20.900161 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:20.880704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:60827)
0806 01:04:20.881417 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:20.881463 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:20.883056 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:20.883799 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:20.883811 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:20.884611 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:04:20.885460 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:20.885488 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:20.886718 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:20.886726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:20.888138 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:20.888151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:20.889264 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:20.889270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:20.889461 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:20.890141 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:04:20.890164 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:04:20.892876 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:04:20.895755 (+ 2879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:20.895765 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:20.895783 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:20.896391 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:20.896707 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:20.896713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:20.896719 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:20.896952 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:20.897411 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:04:20.897420 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:04:20.897869 (+ 449us) client_negotiation.cc:769] Sending connection context
0806 01:04:20.898174 (+ 305us) client_negotiation.cc:240] Negotiation successful
0806 01:04:20.898755 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":260,"threads_started":1}
I20260806 01:04:20.901844 28831 heartbeater.cc:347] Connected to a master server at 127.5.108.190:34403
I20260806 01:04:20.902180 28831 heartbeater.cc:464] Registering TS with master...
I20260806 01:04:20.903056 28831 heartbeater.cc:511] Master 127.5.108.190:34403 requested a full tablet report, sending...
I20260806 01:04:20.905534 28540 ts_manager.cc:194] Registered new tserver with Master: 06aa81588cf24f76a8529c4b8d74823a (127.5.108.129:39047)
I20260806 01:04:20.907727 28540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:60827
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:04:20.952317 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34403
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:04:21.272501 28844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:21.272820 28844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:21.272948 28844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:21.283638 28844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:21.283782 28844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:21.283943 28844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:04:21.292835 28844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:21.293998 28844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:21.295221 28844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:21.307840 28852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:21.307895 28856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:21.307927 28851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:21.308763 28844 server_base.cc:1034] running on GCE node
I20260806 01:04:21.309333 28844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:21.310132 28844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:21.311321 28844 hybrid_clock.cc:648] HybridClock initialized: now 1785978261311285 us; error 27 us; skew 500 ppm
I20260806 01:04:21.311674 28844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:21 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978261, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:21.314448 28844 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:04:21.315773 28844 webserver.cc:460] Webserver started at http://127.5.108.130:46071/ using document root <none> and password file <none>
I20260806 01:04:21.316362 28844 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:21.316497 28844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:21.316792 28844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:04:21.319346 28844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "298613e45bb2449eaeaca69b65abd237"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "e328eeadb16f04b4678d55ae2212f45c"
server_key_iv: "c5cf15eeb69759f6109408f3fa3cb0c7"
server_key_version: "encryptionkey@0"
I20260806 01:04:21.320099 28844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "298613e45bb2449eaeaca69b65abd237"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "e328eeadb16f04b4678d55ae2212f45c"
server_key_iv: "c5cf15eeb69759f6109408f3fa3cb0c7"
server_key_version: "encryptionkey@0"
I20260806 01:04:21.324892 28844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:04:21.328090 28868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:21.329385 28844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:04:21.329537 28844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "298613e45bb2449eaeaca69b65abd237"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "e328eeadb16f04b4678d55ae2212f45c"
server_key_iv: "c5cf15eeb69759f6109408f3fa3cb0c7"
server_key_version: "encryptionkey@0"
I20260806 01:04:21.329728 28844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:21.350764 28844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:21.354454 28844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:21.354713 28844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:21.355633 28844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:21.356791 28844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:21.356889 28844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:21.357023 28844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:21.357116 28844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:21.388571 28844 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:45365
I20260806 01:04:21.388643 29072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:45365 every 8 connection(s)
I20260806 01:04:21.389783 28844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:04:21.390183 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 28844
I20260806 01:04:21.390362 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:04:21.390980 5554 external_mini_cluster.cc:1428] Setting key c902c4879b452e9e4da77f840838de76
Aug 06 01:04:21 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978261, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:04:21.409623 28834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.392433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:39495 (local address 127.5.108.190:34403)
0806 01:04:21.392620 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:21.392626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:21.393845 (+ 1219us) server_negotiation.cc:407] Connection header received
0806 01:04:21.395564 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:21.395569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:21.395667 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:21.395770 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:21.397166 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.397816 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.399258 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.399734 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.399859 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.403599 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:21.403604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:21.403607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:21.403651 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:21.406531 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:21.407175 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:21.407183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:21.407188 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:21.407374 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:21.407772 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:21.407780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:21.407786 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:21.408154 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:21.408303 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:21.408941 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 01:04:21.409191 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:04:21.410179 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.392969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:39495)
0806 01:04:21.393710 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:21.393749 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:21.395284 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:21.395984 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:21.395992 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:21.396489 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:04:21.397008 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.397027 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.397952 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.397957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.399108 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.399121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.399939 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.399946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.400162 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.400796 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:04:21.400829 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:04:21.403396 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:04:21.406679 (+ 3283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:21.406685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:21.406694 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:21.407062 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:21.407499 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:21.407502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:21.407505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:21.407668 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:21.408270 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:04:21.408275 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:04:21.408604 (+ 329us) client_negotiation.cc:769] Sending connection context
0806 01:04:21.408817 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 01:04:21.409179 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":324,"threads_started":1}
I20260806 01:04:21.411518 29073 heartbeater.cc:347] Connected to a master server at 127.5.108.190:34403
I20260806 01:04:21.411830 29073 heartbeater.cc:464] Registering TS with master...
I20260806 01:04:21.412726 29073 heartbeater.cc:511] Master 127.5.108.190:34403 requested a full tablet report, sending...
I20260806 01:04:21.414214 28540 ts_manager.cc:194] Registered new tserver with Master: 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365)
I20260806 01:04:21.415567 28540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:39495
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:04:21.453679 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34403
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:04:21.778043 29079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:21.778297 29079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:21.778383 29079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:21.785115 29079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:21.785228 29079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:21.785413 29079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:04:21.794271 29079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:21.795398 29079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:21.796612 29079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:21.809451 29092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:21.809423 29094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:21.809799 29087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:21.810035 29079 server_base.cc:1034] running on GCE node
I20260806 01:04:21.810639 29079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:21.811563 29079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:21.812789 29079 hybrid_clock.cc:648] HybridClock initialized: now 1785978261812742 us; error 40 us; skew 500 ppm
I20260806 01:04:21.813067 29079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:21 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978261, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:21.816040 29079 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:04:21.817232 29079 webserver.cc:460] Webserver started at http://127.5.108.131:33821/ using document root <none> and password file <none>
I20260806 01:04:21.817858 29079 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:21.817981 29079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:21.818274 29079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:04:21.820937 29079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "04bc7c319ace448daa63d3c2ce33a93e"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "9bcbc4b0523c5ce6e9121b18f4431b73"
server_key_iv: "4567fe82ffdc91a2387e3c6ebfe3c59f"
server_key_version: "encryptionkey@0"
I20260806 01:04:21.821674 29079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "04bc7c319ace448daa63d3c2ce33a93e"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "9bcbc4b0523c5ce6e9121b18f4431b73"
server_key_iv: "4567fe82ffdc91a2387e3c6ebfe3c59f"
server_key_version: "encryptionkey@0"
I20260806 01:04:21.826324 29079 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:04:21.829736 29105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:21.830798 29079 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 01:04:21.830952 29079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "04bc7c319ace448daa63d3c2ce33a93e"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "9bcbc4b0523c5ce6e9121b18f4431b73"
server_key_iv: "4567fe82ffdc91a2387e3c6ebfe3c59f"
server_key_version: "encryptionkey@0"
I20260806 01:04:21.831125 29079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:21.845626 29079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:21.849105 29079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:21.849355 29079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:21.850155 29079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:21.851311 29079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:21.851431 29079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:21.851572 29079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:21.851622 29079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:21.883239 29079 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:40097
I20260806 01:04:21.883301 29282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:40097 every 8 connection(s)
I20260806 01:04:21.884420 29079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:04:21.884828 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 29079
I20260806 01:04:21.885041 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:04:21.885627 5554 external_mini_cluster.cc:1428] Setting key b1e1ee9a781676ccc3383132de693159
Aug 06 01:04:21 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978261, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:04:21.903123 28834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.887005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:33951 (local address 127.5.108.190:34403)
0806 01:04:21.887189 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:21.887194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:21.888401 (+ 1207us) server_negotiation.cc:407] Connection header received
0806 01:04:21.890245 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:21.890250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:21.890334 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:21.890430 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:21.891872 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.892619 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.894043 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.894513 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.894637 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.897773 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:21.897783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:21.897787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:21.897816 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:21.900047 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:21.900874 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:21.900883 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:21.900885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:21.901006 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:21.901449 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:21.901453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:21.901455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:21.901746 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:21.901906 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:21.902574 (+ 668us) server_negotiation.cc:299] Negotiation successful
0806 01:04:21.902786 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 01:04:21.903981 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.887476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:33951)
0806 01:04:21.888255 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:21.888296 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:21.890018 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:21.890651 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:21.890658 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:21.891097 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:04:21.891716 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.891733 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.892772 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.892775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.893915 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.893926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.894728 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.894734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.894940 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.895531 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:04:21.895552 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:04:21.897584 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:04:21.900281 (+ 2697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:21.900291 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:21.900305 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:21.900721 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:21.901120 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:21.901125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:21.901128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:21.901322 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:21.901876 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:04:21.901883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:04:21.902189 (+ 306us) client_negotiation.cc:769] Sending connection context
0806 01:04:21.902435 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 01:04:21.902902 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":330,"threads_started":1}
I20260806 01:04:21.905254 29286 heartbeater.cc:347] Connected to a master server at 127.5.108.190:34403
I20260806 01:04:21.905486 29286 heartbeater.cc:464] Registering TS with master...
I20260806 01:04:21.906096 29286 heartbeater.cc:511] Master 127.5.108.190:34403 requested a full tablet report, sending...
I20260806 01:04:21.907338 28540 ts_manager.cc:194] Registered new tserver with Master: 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097)
I20260806 01:04:21.908174 28540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:33951
I20260806 01:04:21.911350 28831 heartbeater.cc:503] Master 127.5.108.190:34403 was elected leader, sending a full tablet report...
I20260806 01:04:21.913168 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:04:21.929530 28834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.917520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.5.108.190:34403)
0806 01:04:21.917715 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:21.917723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:21.918064 (+ 341us) server_negotiation.cc:407] Connection header received
0806 01:04:21.918199 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:21.918203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:21.918261 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:21.918338 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:21.919713 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.920373 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.921653 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.922043 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.922140 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.924565 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:21.924569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:21.924573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:21.924600 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:21.926860 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:21.927590 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:21.927593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:21.927595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:21.927776 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:21.928174 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:21.928178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:21.928185 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:21.928469 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:21.928716 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:21.929057 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 01:04:21.929255 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:04:21.933396 28539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:04:21.940289 28539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:04:21.962967 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.951657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47096)
0806 01:04:21.952593 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:21.952639 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:21.952850 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:21.955374 (+ 2524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:21.955403 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:21.955435 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:21.955956 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.955974 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.957954 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.957959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.960344 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.960359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.961949 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.961958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.962111 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.962147 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:04:21.962271 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 01:04:21.962468 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":107,"thread_start_us":419,"threads_started":1}
I20260806 01:04:21.963150 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.952858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:45365 (local address 127.0.0.1:55356)
0806 01:04:21.955360 (+ 2502us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:21.955403 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:21.955523 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:21.956276 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:21.956280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:21.956308 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:21.956679 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.956694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.958486 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.958491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.960171 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.960183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.962375 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.962381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.962560 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.962596 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:04:21.962685 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 01:04:21.962781 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2362,"mutex_wait_us":12,"thread_start_us":2212,"threads_started":1}
I20260806 01:04:21.963361 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.952974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36984 (local address 127.5.108.131:40097)
0806 01:04:21.955015 (+ 2041us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:21.955022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:21.955051 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:04:21.955147 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:21.955155 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:21.955393 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:21.956197 (+ 804us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:21.956499 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.957609 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.960497 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.961768 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.962657 (+ 889us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.962857 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:21.962919 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 01:04:21.963053 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1883,"thread_start_us":1739,"threads_started":1}
I20260806 01:04:21.963699 29303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.951983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.5.108.129:39047)
0806 01:04:21.954292 (+ 2309us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:21.954302 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:21.954335 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:04:21.954875 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:21.954882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:21.955136 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:21.956105 (+ 969us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:21.956348 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.957816 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.960528 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.961408 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.961558 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.961802 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:21.963281 (+ 1479us) server_negotiation.cc:299] Negotiation successful
0806 01:04:21.963457 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2147,"thread_start_us":1995,"threads_started":1}
I20260806 01:04:21.962996 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.952405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:40097 (local address 127.0.0.1:36984)
0806 01:04:21.953549 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:21.953570 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:21.953703 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:21.955615 (+ 1912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:21.955624 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:21.955645 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:21.956096 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.956108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.957743 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.957750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.960344 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:21.960359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.962014 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.962022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:21.962217 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.962254 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:04:21.962330 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 01:04:21.962490 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":40,"thread_start_us":841,"threads_started":1}
I20260806 01:04:21.963944 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:21.953299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.5.108.130:45365)
0806 01:04:21.954559 (+ 1260us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:21.954568 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:21.955654 (+ 1086us) server_negotiation.cc:407] Connection header received
0806 01:04:21.955768 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:21.955777 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:21.955986 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:21.956200 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:21.956884 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.958310 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.960315 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:21.961734 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:21.961890 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:21.963521 (+ 1631us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:21.963591 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 01:04:21.963706 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1111,"thread_start_us":1008,"threads_started":1}
I20260806 01:04:21.967226 29175 tablet_service.cc:1467] Processing CreateTablet for tablet 5176a599dad04d28b6e630d89a0e7bbc (DEFAULT_TABLE table=test-table [id=a6b51e22ec4545c59e667d0d09045045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:04:21.967294 28735 tablet_service.cc:1467] Processing CreateTablet for tablet 5176a599dad04d28b6e630d89a0e7bbc (DEFAULT_TABLE table=test-table [id=a6b51e22ec4545c59e667d0d09045045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:04:21.967294 28960 tablet_service.cc:1467] Processing CreateTablet for tablet 5176a599dad04d28b6e630d89a0e7bbc (DEFAULT_TABLE table=test-table [id=a6b51e22ec4545c59e667d0d09045045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:04:21.968382 29175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5176a599dad04d28b6e630d89a0e7bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:21.968452 28735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5176a599dad04d28b6e630d89a0e7bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:21.968472 28960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5176a599dad04d28b6e630d89a0e7bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:21.978521 29309 tablet_bootstrap.cc:492] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: Bootstrap starting.
I20260806 01:04:21.981596 29310 tablet_bootstrap.cc:492] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: Bootstrap starting.
I20260806 01:04:21.981864 29309 tablet_bootstrap.cc:654] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:21.982645 29313 tablet_bootstrap.cc:492] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: Bootstrap starting.
I20260806 01:04:21.983817 29309 log.cc:824] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:21.985359 29310 tablet_bootstrap.cc:654] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:21.985852 29313 tablet_bootstrap.cc:654] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:21.986589 29310 log.cc:824] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:21.986825 29313 log.cc:824] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:21.987082 29309 tablet_bootstrap.cc:492] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: No bootstrap required, opened a new log
I20260806 01:04:21.987433 29309 ts_tablet_manager.cc:1397] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 01:04:21.989491 29310 tablet_bootstrap.cc:492] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: No bootstrap required, opened a new log
I20260806 01:04:21.989733 29310 ts_tablet_manager.cc:1397] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 01:04:21.989789 29313 tablet_bootstrap.cc:492] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: No bootstrap required, opened a new log
I20260806 01:04:21.989998 29313 ts_tablet_manager.cc:1397] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260806 01:04:21.994066 29309 raft_consensus.cc:348] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } }
I20260806 01:04:21.994401 29309 raft_consensus.cc:374] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:21.994498 29309 raft_consensus.cc:729] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04bc7c319ace448daa63d3c2ce33a93e, State: Initialized, Role: FOLLOWER
I20260806 01:04:21.995010 29309 consensus_queue.cc:260] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } }
I20260806 01:04:21.995985 29286 heartbeater.cc:503] Master 127.5.108.190:34403 was elected leader, sending a full tablet report...
I20260806 01:04:21.996539 29309 ts_tablet_manager.cc:1428] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: Time spent starting tablet: real 0.009s user 0.000s sys 0.008s
I20260806 01:04:21.999460 29310 raft_consensus.cc:348] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } }
I20260806 01:04:22.000154 29310 raft_consensus.cc:374] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:22.000270 29310 raft_consensus.cc:729] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aa81588cf24f76a8529c4b8d74823a, State: Initialized, Role: FOLLOWER
I20260806 01:04:22.000584 29313 raft_consensus.cc:348] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } }
I20260806 01:04:22.000974 29313 raft_consensus.cc:374] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:22.000787 29310 consensus_queue.cc:260] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } }
I20260806 01:04:22.001057 29313 raft_consensus.cc:729] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 298613e45bb2449eaeaca69b65abd237, State: Initialized, Role: FOLLOWER
I20260806 01:04:22.001642 29313 consensus_queue.cc:260] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } }
I20260806 01:04:22.002550 29310 ts_tablet_manager.cc:1428] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260806 01:04:22.003422 29073 heartbeater.cc:503] Master 127.5.108.190:34403 was elected leader, sending a full tablet report...
I20260806 01:04:22.003976 29313 ts_tablet_manager.cc:1428] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
W20260806 01:04:22.129848 28833 tablet.cc:2369] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:04:22.136183 29288 tablet.cc:2369] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:04:22.141724 29074 tablet.cc:2369] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:04:22.153411 29330 raft_consensus.cc:482] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:04:22.153734 29330 raft_consensus.cc:504] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } }
I20260806 01:04:22.155632 29330 leader_election.cc:288] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06aa81588cf24f76a8529c4b8d74823a (127.5.108.129:39047), 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097)
I20260806 01:04:22.162668 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:22.156286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:35597 (local address 127.5.108.131:40097)
0806 01:04:22.156515 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:22.156523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:22.157038 (+ 515us) server_negotiation.cc:407] Connection header received
0806 01:04:22.157272 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:22.157276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:22.157332 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:22.157420 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:22.158297 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.159051 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.160818 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.161590 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.161869 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:22.161950 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:22.162268 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 01:04:22.162392 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:04:22.162686 29303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:22.156179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:47345 (local address 127.5.108.129:39047)
0806 01:04:22.156434 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:22.156442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:22.157037 (+ 595us) server_negotiation.cc:407] Connection header received
0806 01:04:22.157261 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:22.157265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:22.157328 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:22.157426 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:22.158355 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.159048 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.160849 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.161686 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.161895 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:22.161972 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:22.162301 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 01:04:22.162428 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 01:04:22.162894 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:22.156010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.5.108.130:47345)
0806 01:04:22.156918 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:22.156939 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:22.157111 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:22.157440 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:22.157445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:22.157500 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:22.158191 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:22.158200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.159193 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.159200 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:22.160687 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:22.160699 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.161935 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.161941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:22.162147 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:22.162170 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 01:04:22.162271 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 01:04:22.162390 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":57,"spinlock_wait_cycles":768,"thread_start_us":616,"threads_started":1}
I20260806 01:04:22.163224 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:22.156069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:40097 (local address 127.5.108.130:35597)
0806 01:04:22.156942 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:22.156955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:22.157135 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:22.157441 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:22.157444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:22.157503 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:22.158144 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:22.158156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.159184 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.159189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:22.160671 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:22.160683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.161907 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.161917 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:22.162076 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:22.162121 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 01:04:22.162231 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 01:04:22.162355 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":5,"spinlock_wait_cycles":79104,"thread_start_us":432,"threads_started":1}
I20260806 01:04:22.163892 28766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5176a599dad04d28b6e630d89a0e7bbc" candidate_uuid: "298613e45bb2449eaeaca69b65abd237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aa81588cf24f76a8529c4b8d74823a" is_pre_election: true
I20260806 01:04:22.164147 29208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5176a599dad04d28b6e630d89a0e7bbc" candidate_uuid: "298613e45bb2449eaeaca69b65abd237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04bc7c319ace448daa63d3c2ce33a93e" is_pre_election: true
I20260806 01:04:22.164397 28766 raft_consensus.cc:2454] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 298613e45bb2449eaeaca69b65abd237 in term 0.
I20260806 01:04:22.164635 29208 raft_consensus.cc:2454] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 298613e45bb2449eaeaca69b65abd237 in term 0.
I20260806 01:04:22.165126 28874 leader_election.cc:302] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aa81588cf24f76a8529c4b8d74823a, 298613e45bb2449eaeaca69b65abd237; no voters:
I20260806 01:04:22.165621 29330 raft_consensus.cc:2781] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:04:22.165771 29330 raft_consensus.cc:482] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:04:22.165881 29330 raft_consensus.cc:3037] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:22.168020 29330 raft_consensus.cc:504] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } }
I20260806 01:04:22.168596 29330 leader_election.cc:288] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [CANDIDATE]: Term 1 election: Requested vote from peers 06aa81588cf24f76a8529c4b8d74823a (127.5.108.129:39047), 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097)
I20260806 01:04:22.169031 28766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5176a599dad04d28b6e630d89a0e7bbc" candidate_uuid: "298613e45bb2449eaeaca69b65abd237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aa81588cf24f76a8529c4b8d74823a"
I20260806 01:04:22.169032 29208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5176a599dad04d28b6e630d89a0e7bbc" candidate_uuid: "298613e45bb2449eaeaca69b65abd237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04bc7c319ace448daa63d3c2ce33a93e"
I20260806 01:04:22.169209 28766 raft_consensus.cc:3037] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:22.169224 29208 raft_consensus.cc:3037] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:22.171545 29208 raft_consensus.cc:2454] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 298613e45bb2449eaeaca69b65abd237 in term 1.
I20260806 01:04:22.171591 28766 raft_consensus.cc:2454] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 298613e45bb2449eaeaca69b65abd237 in term 1.
I20260806 01:04:22.171962 28869 leader_election.cc:302] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04bc7c319ace448daa63d3c2ce33a93e, 298613e45bb2449eaeaca69b65abd237; no voters:
I20260806 01:04:22.172259 29330 raft_consensus.cc:2781] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:04:22.172674 29330 raft_consensus.cc:686] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 1 LEADER]: Becoming Leader. State: Replica: 298613e45bb2449eaeaca69b65abd237, State: Running, Role: LEADER
I20260806 01:04:22.173063 29330 consensus_queue.cc:237] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } }
I20260806 01:04:22.177373 28540 catalog_manager.cc:5495] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 reported cstate change: term changed from 0 to 1, leader changed from <none> to 298613e45bb2449eaeaca69b65abd237 (127.5.108.130). New cstate: current_term: 1 leader_uuid: "298613e45bb2449eaeaca69b65abd237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } health_report { overall_health: UNKNOWN } } }
I20260806 01:04:22.198916 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:22.193968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55366 (local address 127.5.108.130:45365)
0806 01:04:22.194144 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:22.194148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:22.194163 (+ 15us) server_negotiation.cc:407] Connection header received
0806 01:04:22.194330 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:22.194334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:22.194399 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:22.194514 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:04:22.194863 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.195589 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.196931 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.197292 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.197372 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:22.197932 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:04:22.198378 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:04:22.198478 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:22.198590 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 01:04:22.198648 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:04:22.206647 28766 raft_consensus.cc:1261] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Refusing update from remote peer 298613e45bb2449eaeaca69b65abd237: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:04:22.206655 29208 raft_consensus.cc:1261] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [term 1 FOLLOWER]: Refusing update from remote peer 298613e45bb2449eaeaca69b65abd237: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:04:22.207777 29333 consensus_queue.cc:1035] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:04:22.208103 29330 consensus_queue.cc:1035] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:04:22.231873 28540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:04:22.232167 28540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:04:22.235838 28540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Sending DeleteTablet for 3 replicas of tablet 5176a599dad04d28b6e630d89a0e7bbc
I20260806 01:04:22.236855 28735 tablet_service.cc:1514] Processing DeleteTablet for tablet 5176a599dad04d28b6e630d89a0e7bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:04:22 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:47096
I20260806 01:04:22.236967 28960 tablet_service.cc:1514] Processing DeleteTablet for tablet 5176a599dad04d28b6e630d89a0e7bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:04:22 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:55356
I20260806 01:04:22.237012 29175 tablet_service.cc:1514] Processing DeleteTablet for tablet 5176a599dad04d28b6e630d89a0e7bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:04:22 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:36984
I20260806 01:04:22.237620 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 28631
I20260806 01:04:22.252280 29358 tablet_replica.cc:330] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: stopping tablet replica
I20260806 01:04:22.252648 29358 raft_consensus.cc:2229] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:04:22.253072 29358 raft_consensus.cc:2258] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:04:22.255187 29353 tablet_replica.cc:330] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: stopping tablet replica
I20260806 01:04:22.255802 29356 tablet_replica.cc:330] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: stopping tablet replica
I20260806 01:04:22.256162 29356 raft_consensus.cc:2229] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:04:22.256383 29358 ts_tablet_manager.cc:1905] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:04:22.256785 29356 raft_consensus.cc:2258] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:04:22.257269 29353 raft_consensus.cc:2229] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:04:22.257740 29353 raft_consensus.cc:2258] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:04:22.259635 29353 ts_tablet_manager.cc:1905] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:04:22.260599 29356 ts_tablet_manager.cc:1905] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:04:22.262750 29358 ts_tablet_manager.cc:1918] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:04:22.262900 29358 log.cc:1196] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/5176a599dad04d28b6e630d89a0e7bbc
I20260806 01:04:22.263485 29358 ts_tablet_manager.cc:1939] T 5176a599dad04d28b6e630d89a0e7bbc P 04bc7c319ace448daa63d3c2ce33a93e: Deleting consensus metadata
I20260806 01:04:22.265283 28513 catalog_manager.cc:4841] TS 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097): tablet 5176a599dad04d28b6e630d89a0e7bbc (table test-table [id=a6b51e22ec4545c59e667d0d09045045]) successfully deleted
I20260806 01:04:22.266387 29353 ts_tablet_manager.cc:1918] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:04:22.266538 29353 log.cc:1196] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/5176a599dad04d28b6e630d89a0e7bbc
I20260806 01:04:22.266938 29353 ts_tablet_manager.cc:1939] T 5176a599dad04d28b6e630d89a0e7bbc P 06aa81588cf24f76a8529c4b8d74823a: Deleting consensus metadata
I20260806 01:04:22.267683 29356 ts_tablet_manager.cc:1918] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:04:22.267836 29356 log.cc:1196] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/5176a599dad04d28b6e630d89a0e7bbc
I20260806 01:04:22.268256 29356 ts_tablet_manager.cc:1939] T 5176a599dad04d28b6e630d89a0e7bbc P 298613e45bb2449eaeaca69b65abd237: Deleting consensus metadata
I20260806 01:04:22.268272 28517 catalog_manager.cc:4841] TS 06aa81588cf24f76a8529c4b8d74823a (127.5.108.129:39047): tablet 5176a599dad04d28b6e630d89a0e7bbc (table test-table [id=a6b51e22ec4545c59e667d0d09045045]) successfully deleted
I20260806 01:04:22.269663 28517 catalog_manager.cc:4841] TS 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365): tablet 5176a599dad04d28b6e630d89a0e7bbc (table test-table [id=a6b51e22ec4545c59e667d0d09045045]) successfully deleted
Aug 06 01:04:22 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:04:22.279821 29303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:22.239045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47106 (local address 127.5.108.129:39047)
0806 01:04:22.239242 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:22.239250 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:22.247935 (+ 8685us) server_negotiation.cc:407] Connection header received
0806 01:04:22.248362 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:22.248366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:22.248446 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:22.248516 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:22.250598 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.252038 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.254677 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.255336 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.255495 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:22.274260 (+ 18765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:22.274271 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:22.274283 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:22.274319 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:22.276937 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:22.277701 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:22.277707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:22.277709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:22.277833 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:22.278236 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:22.278241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:22.278244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:22.278589 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:22.278891 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:22.279226 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 01:04:22.279453 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:04:22.280464 28824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:22.367172 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 28631
I20260806 01:04:22.394037 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 28844
Aug 06 01:04:22 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:04:22.414032 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:22.394934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.5.108.130:45365)
0806 01:04:22.395128 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:22.395134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:22.395157 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:04:22.395232 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:22.395237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:22.395320 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:22.395481 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:22.397321 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.398466 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.400555 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.401126 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.401229 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:22.407045 (+ 5816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:22.407059 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:22.407077 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:22.407118 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:22.410980 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:22.411884 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:22.411893 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:22.411897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:22.412064 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:22.412529 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:22.412534 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:22.412538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:22.412965 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:22.413270 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:22.413421 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 01:04:22.413651 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:04:22.414796 29059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:22.527295 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 28844
I20260806 01:04:22.554309 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 29079
Aug 06 01:04:22 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:04:22.568430 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:22.555017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.5.108.131:40097)
0806 01:04:22.555195 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:22.555201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:22.555218 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:04:22.555422 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:22.555428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:22.555498 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:22.555578 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:22.556841 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.557498 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.558709 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:22.559046 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:22.559128 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:22.563196 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:22.563207 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:22.563222 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:22.563256 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:22.565716 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:22.566287 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:22.566293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:22.566295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:22.566412 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:22.566721 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:22.566724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:22.566726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:22.567057 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:22.567255 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:22.568009 (+ 754us) server_negotiation.cc:299] Negotiation successful
0806 01:04:22.568161 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:04:22.569083 29272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:22.653488 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 29079
I20260806 01:04:22.680370 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 28485
I20260806 01:04:22.681571 28583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:22.788203 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 28485
I20260806 01:04:42.817643 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34403
--webserver_interface=127.5.108.190
--webserver_port=32913
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:34403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:04:43.146030 29367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:43.146299 29367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:43.146386 29367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:43.153419 29367 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 01:04:43.153509 29367 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 01:04:43.153548 29367 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 01:04:43.153594 29367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:04:43.153646 29367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:43.153714 29367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:43.153766 29367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:04:43.153820 29367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:04:43.163059 29367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:34403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=32913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:43.164374 29367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:43.165654 29367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:43.191227 29375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:43.191184 29376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:43.191146 29380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:43.191895 29367 server_base.cc:1034] running on GCE node
I20260806 01:04:43.192793 29367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:43.194336 29367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:43.195590 29367 hybrid_clock.cc:648] HybridClock initialized: now 1785978283195531 us; error 51 us; skew 500 ppm
I20260806 01:04:43.195982 29367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:43 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978283, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:43.204033 29367 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:04:43.205639 29367 webserver.cc:460] Webserver started at http://127.5.108.190:32913/ using document root <none> and password file <none>
I20260806 01:04:43.206357 29367 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:43.206497 29367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:43.212255 29367 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 01:04:43.215286 29391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:43.216668 29367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:04:43.216852 29367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "cb4f642f13b8422f9615c795e0cee180"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "083b5f0825c7de14fc346563ef4ab720"
server_key_iv: "02bb50c17a42c423cfc466e9e55f1502"
server_key_version: "encryptionkey@0"
I20260806 01:04:43.217793 29367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:43.235423 29367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:43.239017 29367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:43.239297 29367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:43.257344 29367 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:34403
I20260806 01:04:43.257390 29468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:34403 every 8 connection(s)
I20260806 01:04:43.258665 29367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:04:43.265213 29473 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:04:43.267168 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 29367
I20260806 01:04:43.268141 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:39047
--local_ip_for_outbound_sockets=127.5.108.129
--tserver_master_addrs=127.5.108.190:34403
--webserver_port=36645
--webserver_interface=127.5.108.129
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:04:43.268788 29473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Bootstrap starting.
I20260806 01:04:43.290863 29473 log.cc:824] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:43.302701 29473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:04:43.303247 29473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Bootstrap complete.
I20260806 01:04:43.312136 29473 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:04:43.312973 29473 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb4f642f13b8422f9615c795e0cee180, State: Initialized, Role: FOLLOWER
I20260806 01:04:43.313628 29473 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:04:43.313894 29473 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:04:43.314018 29473 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:04:43.314205 29473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:04:43.317826 29473 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:04:43.318383 29473 leader_election.cc:302] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb4f642f13b8422f9615c795e0cee180; no voters:
I20260806 01:04:43.319051 29473 leader_election.cc:288] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:04:43.319187 29477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:04:43.320411 29477 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 2 LEADER]: Becoming Leader. State: Replica: cb4f642f13b8422f9615c795e0cee180, State: Running, Role: LEADER
I20260806 01:04:43.321020 29477 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:04:43.321357 29473 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:04:43.329777 29479 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb4f642f13b8422f9615c795e0cee180. Latest consensus state: current_term: 2 leader_uuid: "cb4f642f13b8422f9615c795e0cee180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } } }
I20260806 01:04:43.330323 29479 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: This master's current role is: LEADER
I20260806 01:04:43.337826 29489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:04:43.337798 29478 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cb4f642f13b8422f9615c795e0cee180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } } }
I20260806 01:04:43.338210 29478 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: This master's current role is: LEADER
I20260806 01:04:43.345750 29489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:04:43.347162 29489 catalog_manager.cc:1259] Loaded cluster ID: c52bc34ea3fa4711977ffde653f039b1
I20260806 01:04:43.347313 29489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:04:43.351558 29489 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:04:43.353229 29489 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Loaded TSK: 0
I20260806 01:04:43.381695 29489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Generated new TSK 1
I20260806 01:04:43.382028 29489 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:04:43.622170 29475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:43.622721 29475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:43.622916 29475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:43.630219 29475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:43.630398 29475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:43.630577 29475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:04:43.639660 29475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:39047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=36645
--tserver_master_addrs=127.5.108.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:43.640944 29475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:43.642191 29475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:43.656750 29509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:43.656750 29510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:43.657768 29513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:43.657958 29475 server_base.cc:1034] running on GCE node
I20260806 01:04:43.658746 29475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:43.659968 29475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:43.661226 29475 hybrid_clock.cc:648] HybridClock initialized: now 1785978283661161 us; error 51 us; skew 500 ppm
I20260806 01:04:43.661623 29475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:43 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978283, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:43.665657 29475 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:04:43.667133 29475 webserver.cc:460] Webserver started at http://127.5.108.129:36645/ using document root <none> and password file <none>
I20260806 01:04:43.667867 29475 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:43.668023 29475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:43.673626 29475 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 01:04:43.676492 29526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:43.677770 29475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:04:43.677959 29475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "06aa81588cf24f76a8529c4b8d74823a"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "d0a577c211d4ed2a6fe56d0f48110af3"
server_key_iv: "05ec15386a4fd3add5a8e2f8d3a3fc71"
server_key_version: "encryptionkey@0"
I20260806 01:04:43.678916 29475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:43.719203 29475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:43.723745 29475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:43.724116 29475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:43.725143 29475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:43.726444 29475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:43.726573 29475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:43.726730 29475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:43.726833 29475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:43.760043 29475 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:39047
I20260806 01:04:43.760149 29694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:39047 every 8 connection(s)
I20260806 01:04:43.761502 29475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:04:43.769552 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 29475
I20260806 01:04:43.770217 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:45365
--local_ip_for_outbound_sockets=127.5.108.130
--tserver_master_addrs=127.5.108.190:34403
--webserver_port=46071
--webserver_interface=127.5.108.130
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:04:43 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978283, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:04:43.784063 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:43.764356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:46883)
0806 01:04:43.765165 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:43.765210 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:43.766910 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:43.768119 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:43.768125 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:43.768567 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:04:43.769078 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:43.769103 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:43.770723 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:43.770730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:43.772751 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:43.772768 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:43.773759 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:43.773768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:43.774072 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:43.774771 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:04:43.774803 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:04:43.777196 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:04:43.780251 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:43.780260 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:43.780272 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:43.780649 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:43.781065 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:43.781071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:43.781075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:43.781255 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:43.782086 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:04:43.782091 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:04:43.782352 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 01:04:43.782607 (+ 255us) client_negotiation.cc:240] Negotiation successful
0806 01:04:43.783036 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":333,"threads_started":1}
I20260806 01:04:43.784605 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:43.764221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:46883 (local address 127.5.108.190:34403)
0806 01:04:43.764823 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:43.764834 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:43.765378 (+ 544us) server_negotiation.cc:407] Connection header received
0806 01:04:43.767367 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:43.767423 (+ 56us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:43.767735 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:43.767997 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:43.769275 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:43.770557 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:43.772885 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:43.773528 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:43.773672 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:43.777440 (+ 3768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:43.777447 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:43.777458 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:43.777495 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:43.780114 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:43.780798 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:43.780804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:43.780809 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:43.780955 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:43.781366 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:43.781371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:43.781374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:43.781904 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:43.782148 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:43.782855 (+ 707us) server_negotiation.cc:299] Negotiation successful
0806 01:04:43.783328 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":288,"threads_started":1}
I20260806 01:04:43.787179 29696 heartbeater.cc:347] Connected to a master server at 127.5.108.190:34403
I20260806 01:04:43.787535 29696 heartbeater.cc:464] Registering TS with master...
I20260806 01:04:43.788201 29696 heartbeater.cc:511] Master 127.5.108.190:34403 requested a full tablet report, sending...
I20260806 01:04:43.790256 29423 ts_manager.cc:194] Registered new tserver with Master: 06aa81588cf24f76a8529c4b8d74823a (127.5.108.129:39047)
I20260806 01:04:43.791795 29423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:46883
W20260806 01:04:44.097628 29702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:44.097930 29702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:44.098131 29702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:44.104828 29702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:44.104964 29702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:44.105158 29702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:04:44.114684 29702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:45365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=46071
--tserver_master_addrs=127.5.108.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:44.115911 29702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:44.117156 29702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:44.130157 29711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:44.130301 29712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:44.130491 29720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:44.131197 29702 server_base.cc:1034] running on GCE node
I20260806 01:04:44.131873 29702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:44.132697 29702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:44.133926 29702 hybrid_clock.cc:648] HybridClock initialized: now 1785978284133882 us; error 37 us; skew 500 ppm
I20260806 01:04:44.134260 29702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:44 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:44.137391 29702 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:04:44.138749 29702 webserver.cc:460] Webserver started at http://127.5.108.130:46071/ using document root <none> and password file <none>
I20260806 01:04:44.139358 29702 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:44.139571 29702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:44.144410 29702 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:04:44.146839 29730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:44.148059 29702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:04:44.148239 29702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "298613e45bb2449eaeaca69b65abd237"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "e328eeadb16f04b4678d55ae2212f45c"
server_key_iv: "c5cf15eeb69759f6109408f3fa3cb0c7"
server_key_version: "encryptionkey@0"
I20260806 01:04:44.149150 29702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:44.159989 29702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:44.163362 29702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:44.163687 29702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:44.164577 29702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:44.165796 29702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:44.165884 29702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:44.166030 29702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:44.166119 29702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:44.198804 29702 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:45365
I20260806 01:04:44.198916 29907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:45365 every 8 connection(s)
I20260806 01:04:44.200168 29702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:04:44.201282 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 29702
I20260806 01:04:44.201962 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:40097
--local_ip_for_outbound_sockets=127.5.108.131
--tserver_master_addrs=127.5.108.190:34403
--webserver_port=33821
--webserver_interface=127.5.108.131
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:04:44 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:04:44.221501 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.202733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:39291 (local address 127.5.108.190:34403)
0806 01:04:44.202975 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.202982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.204090 (+ 1108us) server_negotiation.cc:407] Connection header received
0806 01:04:44.205949 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.205957 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.206052 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.206171 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:44.207655 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.208798 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.210286 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.211038 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.211189 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.214725 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:44.214734 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:44.214738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:44.214782 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:44.218274 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:44.218970 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:44.218977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:44.218983 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:44.219131 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:44.219601 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:44.219608 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:44.219614 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:44.219979 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:44.220134 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:44.220729 (+ 595us) server_negotiation.cc:299] Negotiation successful
0806 01:04:44.221019 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:04:44.221990 29910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.203111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:39291)
0806 01:04:44.203926 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:44.203968 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:44.205677 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:44.206363 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:44.206370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:44.206853 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:04:44.207420 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.207447 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.208966 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.208970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.210131 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.210143 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.211233 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.211237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.211462 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.212100 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:04:44.212121 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:04:44.214500 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:04:44.218413 (+ 3913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:44.218419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:44.218429 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:44.218835 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:44.219257 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:44.219260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:44.219262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:44.219477 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:44.220112 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:04:44.220116 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:04:44.220371 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 01:04:44.220576 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 01:04:44.220924 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":373,"threads_started":1}
I20260806 01:04:44.223456 29908 heartbeater.cc:347] Connected to a master server at 127.5.108.190:34403
I20260806 01:04:44.223754 29908 heartbeater.cc:464] Registering TS with master...
I20260806 01:04:44.224611 29908 heartbeater.cc:511] Master 127.5.108.190:34403 requested a full tablet report, sending...
I20260806 01:04:44.226214 29423 ts_manager.cc:194] Registered new tserver with Master: 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365)
I20260806 01:04:44.227499 29423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:39291
W20260806 01:04:44.546590 29912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:04:44.546866 29912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:04:44.546965 29912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:04:44.554440 29912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:04:44.554554 29912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:04:44.554675 29912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:04:44.564210 29912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:40097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=33821
--tserver_master_addrs=127.5.108.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:04:44.565609 29912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:04:44.567071 29912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:04:44.580529 29922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:44.580575 29926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:04:44.580977 29921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:04:44.581650 29912 server_base.cc:1034] running on GCE node
I20260806 01:04:44.582305 29912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:04:44.583304 29912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:04:44.584589 29912 hybrid_clock.cc:648] HybridClock initialized: now 1785978284584526 us; error 63 us; skew 500 ppm
I20260806 01:04:44.584914 29912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:04:44 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:04:44.588174 29912 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:04:44.589527 29912 webserver.cc:460] Webserver started at http://127.5.108.131:33821/ using document root <none> and password file <none>
I20260806 01:04:44.590102 29912 fs_manager.cc:359] Metadata directory not provided
I20260806 01:04:44.590222 29912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:04:44.595301 29912 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:04:44.597898 29938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:44.599139 29912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:04:44.599275 29912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "04bc7c319ace448daa63d3c2ce33a93e"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "9bcbc4b0523c5ce6e9121b18f4431b73"
server_key_iv: "4567fe82ffdc91a2387e3c6ebfe3c59f"
server_key_version: "encryptionkey@0"
I20260806 01:04:44.600262 29912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:04:44.621670 29912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:04:44.625741 29912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:04:44.626008 29912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:04:44.626884 29912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:04:44.628178 29912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:04:44.628266 29912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:44.628341 29912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:04:44.628396 29912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:04:44.661130 29912 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:40097
I20260806 01:04:44.661222 30105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:40097 every 8 connection(s)
I20260806 01:04:44.662516 29912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:04:44.665616 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 29912
Aug 06 01:04:44 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:04:44.682574 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.666440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.5.108.190:34403)
0806 01:04:44.667337 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.667341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.667359 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:04:44.667465 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.667469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.667533 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.667621 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:44.669670 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.670427 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.672222 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.672595 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.672680 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.676136 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:44.676140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:44.676143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:44.676170 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:44.679655 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:44.680312 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:44.680318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:44.680321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:44.680490 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:44.680915 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:44.680922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:44.680928 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:44.681314 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:44.681571 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:44.681908 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 01:04:44.682125 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":710,"threads_started":1}
I20260806 01:04:44.685173 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.665082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:42773 (local address 127.5.108.190:34403)
0806 01:04:44.665304 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.665311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.666464 (+ 1153us) server_negotiation.cc:407] Connection header received
0806 01:04:44.669203 (+ 2739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.669206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.669250 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.669320 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:44.671034 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.671832 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.673877 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.674236 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.674324 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.678144 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:44.678150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:44.678154 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:44.678203 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:44.681769 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:44.682509 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:44.682517 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:44.682525 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:44.682705 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:44.683213 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:44.683217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:44.683222 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:44.683627 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:44.683738 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:44.684622 (+ 884us) server_negotiation.cc:299] Negotiation successful
0806 01:04:44.684847 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:04:44.686045 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.665496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:42773)
0806 01:04:44.666305 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:44.666348 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:44.668936 (+ 2588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:44.669615 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:44.669627 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:44.670167 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:04:44.670870 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.670894 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.671983 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.671991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.673731 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.673747 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.674477 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.674486 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.674816 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.675450 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:04:44.675483 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:04:44.677941 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:04:44.681889 (+ 3948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:44.681897 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:44.681910 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:44.682375 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:44.682858 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:04:44.682865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:04:44.682868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:04:44.683090 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:04:44.683772 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:04:44.683781 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:04:44.684196 (+ 415us) client_negotiation.cc:769] Sending connection context
0806 01:04:44.684499 (+ 303us) client_negotiation.cc:240] Negotiation successful
0806 01:04:44.684866 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":367,"threads_started":1}
I20260806 01:04:44.687249 30106 heartbeater.cc:347] Connected to a master server at 127.5.108.190:34403
I20260806 01:04:44.687579 30106 heartbeater.cc:464] Registering TS with master...
I20260806 01:04:44.688246 30106 heartbeater.cc:511] Master 127.5.108.190:34403 requested a full tablet report, sending...
I20260806 01:04:44.689555 29423 ts_manager.cc:194] Registered new tserver with Master: 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097)
I20260806 01:04:44.690318 29423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:42773
I20260806 01:04:44.697602 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:04:44.704344 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.698431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54010 (local address 127.5.108.190:34403)
0806 01:04:44.698596 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.698604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.699015 (+ 411us) server_negotiation.cc:407] Connection header received
0806 01:04:44.699269 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.699281 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.699432 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.699573 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:04:44.700053 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.700743 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.702222 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.702626 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.702754 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.703317 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:04:44.703669 (+ 352us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 01:04:44.704043 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54010: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 01:04:44.704470 29700 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54010: token expired
W20260806 01:04:44.705019 29292 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.108.190:34403)
I20260806 01:04:44.717375 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.705653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.5.108.190:34403)
0806 01:04:44.705803 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.705807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.705822 (+ 15us) server_negotiation.cc:407] Connection header received
0806 01:04:44.705912 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.705915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.705963 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.706035 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:44.707550 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.708211 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.709482 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.709831 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.709913 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.712600 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:44.712604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:44.712607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:44.712634 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:44.715053 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:44.715737 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:44.715744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:44.715748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:44.715867 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:44.716222 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:44.716225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:44.716227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:44.716499 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:44.716676 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:44.716933 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 01:04:44.717069 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:04:44.720489 29423 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:04:44.728273 29423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:04:44.759981 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.739417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:45365 (local address 127.0.0.1:40216)
0806 01:04:44.748867 (+ 9450us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:44.748932 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:44.749140 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:44.750147 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:44.750155 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:44.750188 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:44.750729 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.750748 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.753665 (+ 2917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.753674 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.757138 (+ 3464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.757171 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.758872 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.758880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.759141 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.759312 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 01:04:44.759443 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 01:04:44.759621 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9119,"mutex_wait_us":66,"thread_start_us":10072,"threads_started":1}
I20260806 01:04:44.760041 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.739550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.5.108.130:45365)
0806 01:04:44.749527 (+ 9977us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.749533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.749558 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:04:44.749636 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.749641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.749869 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.751009 (+ 1140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:44.751319 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.753461 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.757404 (+ 3943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.758605 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.759313 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.759525 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:44.759589 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 01:04:44.759759 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9828,"thread_start_us":9703,"threads_started":1}
I20260806 01:04:44.762800 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.739449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:40097 (local address 127.0.0.1:57514)
0806 01:04:44.749748 (+ 10299us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:44.749766 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:44.749874 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:44.751001 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:44.751008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:44.751043 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:44.751536 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.751550 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.754077 (+ 2527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.754086 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.757144 (+ 3058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.757172 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.761964 (+ 4792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.761973 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.762143 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.762177 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:04:44.762256 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 01:04:44.762410 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10168,"mutex_wait_us":120,"thread_start_us":8904,"threads_started":1}
I20260806 01:04:44.764122 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.739960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57514 (local address 127.5.108.131:40097)
0806 01:04:44.748352 (+ 8392us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.748364 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.749994 (+ 1630us) server_negotiation.cc:407] Connection header received
0806 01:04:44.750115 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.750126 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.750382 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.750636 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:44.751798 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.753863 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.757337 (+ 3474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.761748 (+ 4411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.763137 (+ 1389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.763361 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:44.763627 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 01:04:44.763803 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8152,"thread_start_us":7945,"threads_started":1}
I20260806 01:04:44.764923 29804 tablet_service.cc:1467] Processing CreateTablet for tablet ae8ad49f8a0e4d5e92fcdb884b875593 (DEFAULT_TABLE table=test-table [id=64abba54d9b548fab7a24dca2b65f7c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:04:44.766311 29804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8ad49f8a0e4d5e92fcdb884b875593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:44.767429 30010 tablet_service.cc:1467] Processing CreateTablet for tablet ae8ad49f8a0e4d5e92fcdb884b875593 (DEFAULT_TABLE table=test-table [id=64abba54d9b548fab7a24dca2b65f7c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:04:44.768788 30010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8ad49f8a0e4d5e92fcdb884b875593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:44.769716 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.749884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47546)
0806 01:04:44.756606 (+ 6722us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:44.756623 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:44.756729 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:44.758649 (+ 1920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:44.758656 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:44.758687 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:44.759095 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.759107 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.760934 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.760940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.766772 (+ 5832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.766787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.768750 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.768944 (+ 194us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.769120 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.769151 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 01:04:44.769251 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 01:04:44.769365 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6581,"thread_start_us":6917,"threads_started":1}
I20260806 01:04:44.769886 30126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.750204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.5.108.129:39047)
0806 01:04:44.758069 (+ 7865us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.758077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.758104 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:04:44.758198 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.758205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.758429 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.758620 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:44.759287 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.760782 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.766972 (+ 6190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.768448 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.768636 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.768869 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:44.769324 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 01:04:44.769501 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7703,"thread_start_us":7583,"threads_started":1}
I20260806 01:04:44.773262 29596 tablet_service.cc:1467] Processing CreateTablet for tablet ae8ad49f8a0e4d5e92fcdb884b875593 (DEFAULT_TABLE table=test-table [id=64abba54d9b548fab7a24dca2b65f7c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:04:44.774617 29596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8ad49f8a0e4d5e92fcdb884b875593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:04:44.780437 30128 tablet_bootstrap.cc:492] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: Bootstrap starting.
I20260806 01:04:44.783802 30128 tablet_bootstrap.cc:654] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:44.784840 30128 log.cc:824] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:44.787482 30128 tablet_bootstrap.cc:492] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: No bootstrap required, opened a new log
I20260806 01:04:44.787732 30128 ts_tablet_manager.cc:1397] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260806 01:04:44.788054 30129 tablet_bootstrap.cc:492] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: Bootstrap starting.
I20260806 01:04:44.789140 30134 tablet_bootstrap.cc:492] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Bootstrap starting.
I20260806 01:04:44.791231 30129 tablet_bootstrap.cc:654] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:44.792234 30129 log.cc:824] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:44.792320 30134 tablet_bootstrap.cc:654] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Neither blocks nor log segments found. Creating new log.
I20260806 01:04:44.793469 30134 log.cc:824] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:04:44.794687 29696 heartbeater.cc:503] Master 127.5.108.190:34403 was elected leader, sending a full tablet report...
I20260806 01:04:44.795076 30129 tablet_bootstrap.cc:492] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: No bootstrap required, opened a new log
I20260806 01:04:44.795336 30129 ts_tablet_manager.cc:1397] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 01:04:44.796695 30134 tablet_bootstrap.cc:492] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: No bootstrap required, opened a new log
I20260806 01:04:44.796991 30134 ts_tablet_manager.cc:1397] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260806 01:04:44.797653 30128 raft_consensus.cc:348] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:04:44.798096 30128 raft_consensus.cc:374] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:44.798207 30128 raft_consensus.cc:729] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 298613e45bb2449eaeaca69b65abd237, State: Initialized, Role: FOLLOWER
I20260806 01:04:44.798914 30128 consensus_queue.cc:260] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:04:44.803414 29908 heartbeater.cc:503] Master 127.5.108.190:34403 was elected leader, sending a full tablet report...
I20260806 01:04:44.804121 30128 ts_tablet_manager.cc:1428] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260806 01:04:44.805624 30129 raft_consensus.cc:348] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:04:44.806001 30129 raft_consensus.cc:374] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:44.806094 30129 raft_consensus.cc:729] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04bc7c319ace448daa63d3c2ce33a93e, State: Initialized, Role: FOLLOWER
I20260806 01:04:44.806680 30129 consensus_queue.cc:260] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:04:44.807920 30106 heartbeater.cc:503] Master 127.5.108.190:34403 was elected leader, sending a full tablet report...
I20260806 01:04:44.808501 30129 ts_tablet_manager.cc:1428] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260806 01:04:44.808372 30134 raft_consensus.cc:348] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:04:44.808785 30134 raft_consensus.cc:374] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:04:44.808887 30134 raft_consensus.cc:729] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aa81588cf24f76a8529c4b8d74823a, State: Initialized, Role: FOLLOWER
I20260806 01:04:44.809484 30134 consensus_queue.cc:260] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:04:44.811005 30134 ts_tablet_manager.cc:1428] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
W20260806 01:04:44.914189 30108 tablet.cc:2369] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:04:44.923158 30138 raft_consensus.cc:482] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:04:44.923484 30138 raft_consensus.cc:504] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:04:44.925210 30138 leader_election.cc:288] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365), 06aa81588cf24f76a8529c4b8d74823a (127.5.108.129:39047)
I20260806 01:04:44.930823 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.925600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:45365 (local address 127.5.108.131:55647)
0806 01:04:44.925848 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:44.925871 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:44.925997 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:44.926317 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:44.926322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:44.926379 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:44.926654 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.926692 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.927721 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.927730 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.928984 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.928996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.930102 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.930108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.930306 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.930330 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:04:44.930441 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 01:04:44.930564 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":2}
I20260806 01:04:44.930820 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.925784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:55647 (local address 127.5.108.130:45365)
0806 01:04:44.925988 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.925994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.926014 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:04:44.926115 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.926122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.926190 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.926290 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:44.926830 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.927582 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.929155 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.929949 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.930109 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.930178 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:44.930467 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 01:04:44.930580 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:04:44.932018 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.926052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.5.108.131:33423)
0806 01:04:44.927003 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:44.927017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:04:44.927161 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:04:44.927553 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:04:44.927558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:04:44.927581 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:04:44.927974 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.927984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.928988 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.928994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.930294 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:04:44.930305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.931545 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.931549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:04:44.931650 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.931671 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 01:04:44.931739 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 01:04:44.931813 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"thread_start_us":639,"threads_started":1}
I20260806 01:04:44.931877 29838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8ad49f8a0e4d5e92fcdb884b875593" candidate_uuid: "04bc7c319ace448daa63d3c2ce33a93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "298613e45bb2449eaeaca69b65abd237" is_pre_election: true
I20260806 01:04:44.932171 30126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:44.926323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:33423 (local address 127.5.108.129:39047)
0806 01:04:44.926617 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:44.926622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:44.927116 (+ 494us) server_negotiation.cc:407] Connection header received
0806 01:04:44.927318 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:44.927322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:44.927423 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:44.927541 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:04:44.928138 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.928847 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.930462 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:44.931353 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:44.931533 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:44.931603 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:44.931797 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 01:04:44.931930 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 01:04:44.932379 29838 raft_consensus.cc:2454] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04bc7c319ace448daa63d3c2ce33a93e in term 0.
I20260806 01:04:44.933033 29943 leader_election.cc:302] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04bc7c319ace448daa63d3c2ce33a93e, 298613e45bb2449eaeaca69b65abd237; no voters:
I20260806 01:04:44.933059 29629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8ad49f8a0e4d5e92fcdb884b875593" candidate_uuid: "04bc7c319ace448daa63d3c2ce33a93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aa81588cf24f76a8529c4b8d74823a" is_pre_election: true
I20260806 01:04:44.933406 30138 raft_consensus.cc:2781] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:04:44.933561 30138 raft_consensus.cc:482] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:04:44.933681 30138 raft_consensus.cc:3037] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:44.933640 29629 raft_consensus.cc:2454] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04bc7c319ace448daa63d3c2ce33a93e in term 0.
I20260806 01:04:44.936033 30138 raft_consensus.cc:504] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:04:44.936578 30138 leader_election.cc:288] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [CANDIDATE]: Term 1 election: Requested vote from peers 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365), 06aa81588cf24f76a8529c4b8d74823a (127.5.108.129:39047)
I20260806 01:04:44.936980 29838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8ad49f8a0e4d5e92fcdb884b875593" candidate_uuid: "04bc7c319ace448daa63d3c2ce33a93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "298613e45bb2449eaeaca69b65abd237"
I20260806 01:04:44.937044 29629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8ad49f8a0e4d5e92fcdb884b875593" candidate_uuid: "04bc7c319ace448daa63d3c2ce33a93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aa81588cf24f76a8529c4b8d74823a"
I20260806 01:04:44.937157 29838 raft_consensus.cc:3037] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:44.937228 29629 raft_consensus.cc:3037] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:04:44.939249 29838 raft_consensus.cc:2454] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04bc7c319ace448daa63d3c2ce33a93e in term 1.
I20260806 01:04:44.939312 29629 raft_consensus.cc:2454] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04bc7c319ace448daa63d3c2ce33a93e in term 1.
I20260806 01:04:44.939640 29943 leader_election.cc:302] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04bc7c319ace448daa63d3c2ce33a93e, 298613e45bb2449eaeaca69b65abd237; no voters:
I20260806 01:04:44.939913 30138 raft_consensus.cc:2781] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:04:44.940361 30138 raft_consensus.cc:686] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 1 LEADER]: Becoming Leader. State: Replica: 04bc7c319ace448daa63d3c2ce33a93e, State: Running, Role: LEADER
I20260806 01:04:44.940905 30138 consensus_queue.cc:237] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:04:44.945571 29420 catalog_manager.cc:5495] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e reported cstate change: term changed from 0 to 1, leader changed from <none> to 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131). New cstate: current_term: 1 leader_uuid: "04bc7c319ace448daa63d3c2ce33a93e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } health_report { overall_health: UNKNOWN } } }
W20260806 01:04:44.951972 29909 tablet.cc:2369] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:04:45.006090 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.000984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57518 (local address 127.5.108.131:40097)
0806 01:04:45.001158 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:45.001163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:45.001180 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:04:45.001365 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:45.001369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:45.001458 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:45.001586 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:04:45.001932 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:45.002603 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:45.004094 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:45.004582 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:45.004672 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:45.005176 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:04:45.005638 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:04:45.005745 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:45.005825 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 01:04:45.005902 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:04:45.013909 29629 raft_consensus.cc:1261] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Refusing update from remote peer 04bc7c319ace448daa63d3c2ce33a93e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:04:45.013909 29838 raft_consensus.cc:1261] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [term 1 FOLLOWER]: Refusing update from remote peer 04bc7c319ace448daa63d3c2ce33a93e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:04:45.014918 30144 consensus_queue.cc:1035] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [LEADER]: Connected to new peer: Peer: permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:04:45.015302 30138 consensus_queue.cc:1035] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [LEADER]: Connected to new peer: Peer: permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:04:45.018281 29697 tablet.cc:2369] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:04:45.026222 30149 mvcc.cc:205] Tried to move back new op lower bound from 7315367055407177728 to 7315367055121575936. Current Snapshot: MvccSnapshot[applied={T|T < 7315367055407177728}]
I20260806 01:04:45.026893 30150 mvcc.cc:205] Tried to move back new op lower bound from 7315367055407177728 to 7315367055121575936. Current Snapshot: MvccSnapshot[applied={T|T < 7315367055407177728}]
I20260806 01:04:45.031733 30151 mvcc.cc:205] Tried to move back new op lower bound from 7315367055407177728 to 7315367055121575936. Current Snapshot: MvccSnapshot[applied={T|T < 7315367055407177728}]
I20260806 01:04:45.041091 29420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:04:45.041361 29420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:04:45.044756 29420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Sending DeleteTablet for 3 replicas of tablet ae8ad49f8a0e4d5e92fcdb884b875593
I20260806 01:04:45.045908 30010 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8ad49f8a0e4d5e92fcdb884b875593 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:04:45 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:57514
I20260806 01:04:45.046157 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 29475
I20260806 01:04:45.046178 29804 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8ad49f8a0e4d5e92fcdb884b875593 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:04:45 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:40216
I20260806 01:04:45.046586 29596 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8ad49f8a0e4d5e92fcdb884b875593 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:04:45 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:47546
I20260806 01:04:45.047580 30161 tablet_replica.cc:330] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: stopping tablet replica
I20260806 01:04:45.047976 30161 raft_consensus.cc:2229] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:04:45.048447 30161 raft_consensus.cc:2258] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:04:45.050504 30161 ts_tablet_manager.cc:1905] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:04:45.057626 30161 ts_tablet_manager.cc:1918] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:04:45.057858 30161 log.cc:1196] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/ae8ad49f8a0e4d5e92fcdb884b875593
I20260806 01:04:45.058465 30161 ts_tablet_manager.cc:1939] T ae8ad49f8a0e4d5e92fcdb884b875593 P 298613e45bb2449eaeaca69b65abd237: Deleting consensus metadata
I20260806 01:04:45.060302 29398 catalog_manager.cc:4841] TS 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365): tablet ae8ad49f8a0e4d5e92fcdb884b875593 (table test-table [id=64abba54d9b548fab7a24dca2b65f7c5]) successfully deleted
I20260806 01:04:45.063838 30164 tablet_replica.cc:330] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: stopping tablet replica
I20260806 01:04:45.064304 30164 raft_consensus.cc:2229] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:04:45.064898 30164 raft_consensus.cc:2258] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:04:45.065691 30159 tablet_replica.cc:330] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: stopping tablet replica
I20260806 01:04:45.066140 30159 raft_consensus.cc:2229] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:04:45.066833 30159 raft_consensus.cc:2258] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:04:45.067257 30164 ts_tablet_manager.cc:1905] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:04:45.068040 30126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.047193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47562 (local address 127.5.108.129:39047)
0806 01:04:45.047360 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:45.047397 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:45.047418 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:04:45.047471 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:45.047477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:45.047551 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:45.047653 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:45.049642 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:45.051023 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:45.053495 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:45.054332 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:45.054474 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:45.058709 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:45.058727 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:45.058761 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:45.058821 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:45.063462 (+ 4641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:45.064687 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:45.064699 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:45.064705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:45.064930 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:45.065689 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:45.065694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:45.065697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:45.066145 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:45.066448 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:45.067331 (+ 883us) server_negotiation.cc:299] Negotiation successful
0806 01:04:45.067632 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:04:45.068951 29689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:45.071098 30159 ts_tablet_manager.cc:1905] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:04:45.076500 30159 ts_tablet_manager.cc:1918] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:04:45.076689 30159 log.cc:1196] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/ae8ad49f8a0e4d5e92fcdb884b875593
I20260806 01:04:45.077183 30159 ts_tablet_manager.cc:1939] T ae8ad49f8a0e4d5e92fcdb884b875593 P 04bc7c319ace448daa63d3c2ce33a93e: Deleting consensus metadata
I20260806 01:04:45.078593 29392 catalog_manager.cc:4841] TS 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097): tablet ae8ad49f8a0e4d5e92fcdb884b875593 (table test-table [id=64abba54d9b548fab7a24dca2b65f7c5]) successfully deleted
I20260806 01:04:45.174374 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 29475
I20260806 01:04:45.202999 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 29702
W20260806 01:04:45.203806 29398 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.108.129:39047 (error 108)
I20260806 01:04:45.206493 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.206027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47574)
0806 01:04:45.206203 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:45.206325 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 01:04:45.206739 29398 catalog_manager.cc:4584] TS 06aa81588cf24f76a8529c4b8d74823a (127.5.108.129:39047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae8ad49f8a0e4d5e92fcdb884b875593: Network error: Client connection negotiation failed: client connection to 127.5.108.129:39047: connect: Connection refused (error 111)
I20260806 01:04:45.218657 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.204747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.5.108.130:45365)
0806 01:04:45.204949 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:45.204956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:45.205266 (+ 310us) server_negotiation.cc:407] Connection header received
0806 01:04:45.205329 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:45.205334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:45.205408 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:45.205487 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:45.207288 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:45.208019 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:45.209298 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:45.209672 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:45.209798 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:45.212382 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:45.212393 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:45.212405 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:45.212434 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:45.215746 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:45.216378 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:45.216392 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:45.216394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:45.216509 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:45.216857 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:45.216861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:45.216864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:45.217194 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:45.217491 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:45.218293 (+ 802us) server_negotiation.cc:299] Negotiation successful
0806 01:04:45.218435 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:04:45.219295 29903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:45.260543 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.259946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47590)
0806 01:04:45.260188 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:45.260322 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 01:04:45.262006 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.261489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47598)
0806 01:04:45.261734 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:45.261831 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 01:04:45.322674 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.322139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47606)
0806 01:04:45.322368 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:45.322494 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 01:04:45.323966 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.323514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47608)
0806 01:04:45.323696 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:45.323798 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 01:04:45.331785 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 29702
I20260806 01:04:45.355611 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 29912
I20260806 01:04:45.372331 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.356516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.5.108.131:40097)
0806 01:04:45.356818 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 01:04:45.356825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:04:45.356852 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:04:45.356927 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:04:45.356933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:04:45.357014 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:04:45.357149 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:04:45.358526 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:45.359775 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:45.361031 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:04:45.361805 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:04:45.361976 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:04:45.364668 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:04:45.364685 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:04:45.364707 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:04:45.364768 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:04:45.369009 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:45.369792 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:45.369802 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:45.369807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:45.370013 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:04:45.370420 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:04:45.370426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:04:45.370430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:04:45.370967 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:04:45.371366 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0806 01:04:45.371581 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 01:04:45.371819 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260806 01:04:45.373169 30098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:45.404408 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.403811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47614)
0806 01:04:45.404055 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:45.404176 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 01:04:45.405560 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.405072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47626)
0806 01:04:45.405319 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:45.405412 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 01:04:45.491211 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 29912
I20260806 01:04:45.518450 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 29367
I20260806 01:04:45.519663 29464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:04:45.627982 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:04:45.627196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:39047 (local address 127.0.0.1:47636)
0806 01:04:45.627571 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 01:04:45.627723 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212}
I20260806 01:04:45.627996 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 29367
I20260806 01:05:05.657359 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34403
--webserver_interface=127.5.108.190
--webserver_port=32913
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:34403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:05.996423 30172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:05.996677 30172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:05.996837 30172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:06.003753 30172 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 01:05:06.003847 30172 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 01:05:06.003937 30172 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 01:05:06.004029 30172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:06.004120 30172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:06.004236 30172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:06.004314 30172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:06.004429 30172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:06.013908 30172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:34403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=32913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:06.015055 30172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:06.016498 30172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:06.028101 30180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:06.028204 30181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:06.028162 30183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:06.028945 30172 server_base.cc:1034] running on GCE node
I20260806 01:05:06.029606 30172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:06.030834 30172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:06.032078 30172 hybrid_clock.cc:648] HybridClock initialized: now 1785978306032028 us; error 58 us; skew 500 ppm
I20260806 01:05:06.032401 30172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:06 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978306, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:06.041893 30172 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:05:06.043435 30172 webserver.cc:460] Webserver started at http://127.5.108.190:32913/ using document root <none> and password file <none>
I20260806 01:05:06.044111 30172 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:06.044250 30172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:06.049712 30172 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 01:05:06.052474 30191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:06.053664 30172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:06.053858 30172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "cb4f642f13b8422f9615c795e0cee180"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "083b5f0825c7de14fc346563ef4ab720"
server_key_iv: "02bb50c17a42c423cfc466e9e55f1502"
server_key_version: "encryptionkey@0"
I20260806 01:05:06.054786 30172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:06.065780 30172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:06.069607 30172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:06.069908 30172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:06.088796 30172 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:34403
I20260806 01:05:06.088842 30265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:34403 every 8 connection(s)
I20260806 01:05:06.090137 30172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:06.096284 30266 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:05:06.097750 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 30172
I20260806 01:05:06.098805 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:39047
--local_ip_for_outbound_sockets=127.5.108.129
--tserver_master_addrs=127.5.108.190:34403
--webserver_port=36645
--webserver_interface=127.5.108.129
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:05:06.100667 30266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Bootstrap starting.
I20260806 01:05:06.124253 30266 log.cc:824] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:06.139792 30266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:05:06.140340 30266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Bootstrap complete.
I20260806 01:05:06.149070 30266 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:05:06.149896 30266 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb4f642f13b8422f9615c795e0cee180, State: Initialized, Role: FOLLOWER
I20260806 01:05:06.150571 30266 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:05:06.150830 30266 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:06.150949 30266 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:06.151127 30266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 2 FOLLOWER]: Advancing to term 3
I20260806 01:05:06.154680 30266 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:05:06.155161 30266 leader_election.cc:302] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb4f642f13b8422f9615c795e0cee180; no voters:
I20260806 01:05:06.155900 30266 leader_election.cc:288] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 01:05:06.156045 30274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 01:05:06.157428 30274 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [term 3 LEADER]: Becoming Leader. State: Replica: cb4f642f13b8422f9615c795e0cee180, State: Running, Role: LEADER
I20260806 01:05:06.157940 30274 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } }
I20260806 01:05:06.158186 30266 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:06.160811 30276 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb4f642f13b8422f9615c795e0cee180. Latest consensus state: current_term: 3 leader_uuid: "cb4f642f13b8422f9615c795e0cee180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } } }
I20260806 01:05:06.161219 30276 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:06.162221 30275 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cb4f642f13b8422f9615c795e0cee180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4f642f13b8422f9615c795e0cee180" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34403 } } }
I20260806 01:05:06.162551 30275 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:06.163561 30281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:06.171509 30281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:06.172885 30281 catalog_manager.cc:1259] Loaded cluster ID: c52bc34ea3fa4711977ffde653f039b1
I20260806 01:05:06.173051 30281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:06.177002 30281 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:06.178947 30281 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Loaded TSK: 0
I20260806 01:05:06.179136 30281 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cb4f642f13b8422f9615c795e0cee180: Loaded TSK: 1
F20260806 01:05:06.180289 30281 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f59c41b69ad google::LogMessage::Fail() at ??:0
@ 0x7f59c41b988a google::LogMessage::SendToLog() at ??:0
@ 0x7f59c41b64de google::LogMessage::Flush() at ??:0
@ 0x7f59c41b7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f59c7f1a7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f59c7f0973e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f59d9be51c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f59d9be4783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f59d9c702a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f59d9c700c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f59d9cba5e9 std::function<>::operator()() at ??:0
@ 0x7f59d9be8cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f59d9be6c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f59d9c6e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f59d9c6e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f59c6e02345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f59c6e0591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f59c6e05799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f59c6de0eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f59ca0ba6db start_thread at ??:0
@ 0x7f59c305061f clone at ??:0
W20260806 01:05:06.499872 30269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:06.500181 30269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:06.500346 30269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:06.507417 30269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:06.507551 30269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:06.507720 30269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:05:06.517469 30269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:39047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=36645
--tserver_master_addrs=127.5.108.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:06.518692 30269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:06.520002 30269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:06.532985 30342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:06.533038 30344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:06.533072 30341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:06.533799 30269 server_base.cc:1034] running on GCE node
I20260806 01:05:06.534382 30269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:06.535270 30269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:06.536540 30269 hybrid_clock.cc:648] HybridClock initialized: now 1785978306536480 us; error 62 us; skew 500 ppm
I20260806 01:05:06.536873 30269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:06 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978306, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:06.540155 30269 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:05:06.541565 30269 webserver.cc:460] Webserver started at http://127.5.108.129:36645/ using document root <none> and password file <none>
I20260806 01:05:06.542220 30269 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:06.542371 30269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:06.547566 30269 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:05:06.550396 30357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:06.551648 30269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:06.551812 30269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "06aa81588cf24f76a8529c4b8d74823a"
format_stamp: "Formatted at 2026-08-06 01:04:20 on dist-test-slave-x4qh"
server_key: "d0a577c211d4ed2a6fe56d0f48110af3"
server_key_iv: "05ec15386a4fd3add5a8e2f8d3a3fc71"
server_key_version: "encryptionkey@0"
I20260806 01:05:06.552731 30269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:06.591922 30269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:06.595883 30269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:06.596190 30269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:06.597198 30269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:06.599061 30365 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 01:05:06.604636 30269 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 01:05:06.604799 30269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 01:05:06.604961 30269 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 01:05:06.608137 30269 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 01:05:06.608273 30269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:06.608544 30365 tablet_bootstrap.cc:492] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Bootstrap starting.
I20260806 01:05:06.616871 30365 log.cc:824] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:06.632332 30365 tablet_bootstrap.cc:492] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:05:06.633160 30365 tablet_bootstrap.cc:492] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Bootstrap complete.
I20260806 01:05:06.633754 30365 ts_tablet_manager.cc:1397] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Time spent bootstrapping tablet: real 0.026s user 0.020s sys 0.004s
I20260806 01:05:06.644256 30365 raft_consensus.cc:348] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:05:06.645773 30365 raft_consensus.cc:729] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aa81588cf24f76a8529c4b8d74823a, State: Initialized, Role: FOLLOWER
I20260806 01:05:06.648056 30365 consensus_queue.cc:260] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:05:06.652720 30365 ts_tablet_manager.cc:1428] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a: Time spent starting tablet: real 0.019s user 0.013s sys 0.003s
I20260806 01:05:06.653203 30269 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:39047
I20260806 01:05:06.653244 30523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:39047 every 8 connection(s)
I20260806 01:05:06.654378 30269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:05:06.659194 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:06.657472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:60799)
0806 01:05:06.658195 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:06.658422 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":309,"threads_started":1}
W20260806 01:05:06.659834 30524 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:34403 (0 consecutive failures): Network error: Failed to ping master at 127.5.108.190:34403: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
I20260806 01:05:06.660571 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:06.660293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:38653)
0806 01:05:06.660415 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:06.660476 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 01:05:06.661501 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:06.661062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:38661)
0806 01:05:06.661239 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:06.661336 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 01:05:06.661768 30524 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:05:06.661762 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 30269
I20260806 01:05:06.662406 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:45365
--local_ip_for_outbound_sockets=127.5.108.130
--tserver_master_addrs=127.5.108.190:34403
--webserver_port=46071
--webserver_interface=127.5.108.130
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:07.008925 30529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:07.009225 30529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:07.009397 30529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:07.016355 30529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:07.016494 30529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:07.016705 30529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:05:07.026091 30529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:45365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=46071
--tserver_master_addrs=127.5.108.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:07.027359 30529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:07.028662 30529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:07.041530 30541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:07.041602 30537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:07.041646 30536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:07.042421 30529 server_base.cc:1034] running on GCE node
I20260806 01:05:07.043015 30529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:07.043982 30529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:07.045208 30529 hybrid_clock.cc:648] HybridClock initialized: now 1785978307045141 us; error 53 us; skew 500 ppm
I20260806 01:05:07.045529 30529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:07 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978307, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:07.048671 30529 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:05:07.050122 30529 webserver.cc:460] Webserver started at http://127.5.108.130:46071/ using document root <none> and password file <none>
I20260806 01:05:07.050760 30529 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:07.050863 30529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:07.056154 30529 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:07.058846 30553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:07.060216 30529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:05:07.060344 30529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "298613e45bb2449eaeaca69b65abd237"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "e328eeadb16f04b4678d55ae2212f45c"
server_key_iv: "c5cf15eeb69759f6109408f3fa3cb0c7"
server_key_version: "encryptionkey@0"
I20260806 01:05:07.061265 30529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:07.091588 30529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:07.095322 30529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:07.095618 30529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:07.096535 30529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:07.097797 30529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:07.097880 30529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:07.097959 30529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:07.098011 30529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:07.130628 30529 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:45365
I20260806 01:05:07.130720 30725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:45365 every 8 connection(s)
I20260806 01:05:07.132231 30529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:05:07.133117 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 30529
I20260806 01:05:07.133913 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:40097
--local_ip_for_outbound_sockets=127.5.108.131
--tserver_master_addrs=127.5.108.190:34403
--webserver_port=33821
--webserver_interface=127.5.108.131
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:05:07.137938 30730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.135738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:44387)
0806 01:05:07.136600 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:07.136887 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":306,"threads_started":1}
W20260806 01:05:07.138659 30727 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:34403 (0 consecutive failures): Network error: Failed to ping master at 127.5.108.190:34403: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
I20260806 01:05:07.139609 30730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.139167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:49097)
0806 01:05:07.139356 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:07.139467 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 01:05:07.140578 30730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.140158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:52845)
0806 01:05:07.140331 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:07.140426 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:05:07.140844 30727 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:05:07.479072 30731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:07.479310 30731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:07.479471 30731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:07.486713 30731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:07.486831 30731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:07.487013 30731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:05:07.496192 30731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:40097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=33821
--tserver_master_addrs=127.5.108.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:07.497349 30731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:07.498611 30731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:07.513935 30738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:07.513908 30737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:07.514099 30741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:07.515460 30731 server_base.cc:1034] running on GCE node
I20260806 01:05:07.516042 30731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:07.516912 30731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:07.518105 30731 hybrid_clock.cc:648] HybridClock initialized: now 1785978307518055 us; error 47 us; skew 500 ppm
I20260806 01:05:07.518404 30731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:07 dist-test-slave-x4qh krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978307, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:07.521353 30731 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:05:07.522727 30731 webserver.cc:460] Webserver started at http://127.5.108.131:33821/ using document root <none> and password file <none>
I20260806 01:05:07.523463 30731 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:07.523609 30731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:07.528690 30731 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 01:05:07.531476 30761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:07.532678 30731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:05:07.532871 30731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "04bc7c319ace448daa63d3c2ce33a93e"
format_stamp: "Formatted at 2026-08-06 01:04:21 on dist-test-slave-x4qh"
server_key: "9bcbc4b0523c5ce6e9121b18f4431b73"
server_key_iv: "4567fe82ffdc91a2387e3c6ebfe3c59f"
server_key_version: "encryptionkey@0"
I20260806 01:05:07.533818 30731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:07.554231 30731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:07.558216 30731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:07.558485 30731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:07.559486 30731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:07.560776 30731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:07.560933 30731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:07.561097 30731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:07.561190 30731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:07.595677 30731 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:40097
I20260806 01:05:07.595748 30928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:40097 every 8 connection(s)
I20260806 01:05:07.597127 30731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:05:07.601585 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.599914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:40505)
0806 01:05:07.600577 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:07.600806 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":260,"threads_started":1}
W20260806 01:05:07.602224 30931 heartbeater.cc:644] Failed to heartbeat to 127.5.108.190:34403 (0 consecutive failures): Network error: Failed to ping master at 127.5.108.190:34403: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
I20260806 01:05:07.602942 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.602643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:48149)
0806 01:05:07.602788 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:07.602848 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 01:05:07.603740 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.603459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:37573)
0806 01:05:07.603567 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:07.603637 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 01:05:07.603956 30931 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:05:07.605749 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 30731
I20260806 01:05:07.662954 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.662207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:39443)
0806 01:05:07.662668 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:07.662746 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":221,"threads_started":1}
I20260806 01:05:07.795428 30937 raft_consensus.cc:482] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:07.795755 30937 raft_consensus.cc:504] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:05:07.797377 30937 leader_election.cc:288] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365), 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097)
Aug 06 01:05:07 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978306, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
Aug 06 01:05:07 dist-test-slave-x4qh krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978306, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:05:07.823402 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.797851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:39883 (local address 127.5.108.130:45365)
0806 01:05:07.800374 (+ 2523us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:07.800387 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:07.800422 (+ 35us) server_negotiation.cc:407] Connection header received
0806 01:05:07.802448 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:07.802457 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:07.802885 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:07.803258 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:07.805957 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:07.807923 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:07.810658 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:07.811708 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:07.811906 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:07.816954 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:07.816961 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:07.816973 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:07.817006 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:07.819899 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:07.820647 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:07.820653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:07.820659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:07.820816 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:07.821247 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:07.821252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:07.821257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:07.821721 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:07.821923 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:07.822626 (+ 703us) server_negotiation.cc:299] Negotiation successful
0806 01:05:07.822983 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2339,"thread_start_us":2223,"threads_started":1}
I20260806 01:05:07.823421 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.797737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:45365 (local address 127.5.108.129:39883)
0806 01:05:07.798071 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:07.798127 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:07.798785 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:07.803422 (+ 4637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:07.803435 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:07.804469 (+ 1034us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:07.805718 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:07.805748 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:07.808122 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:07.808128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:07.810498 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:07.810511 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:07.811956 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:07.811963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:07.812307 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:07.813235 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:07.813269 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:07.816674 (+ 3405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:07.820037 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:07.820045 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:07.820057 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:07.820509 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:07.820931 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:07.820935 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:07.820939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:07.821116 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:07.821884 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:07.821891 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:07.822154 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 01:05:07.822406 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 01:05:07.822925 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":421,"spinlock_wait_cycles":25728}
I20260806 01:05:07.823419 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.798153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:60591 (local address 127.5.108.131:40097)
0806 01:05:07.800126 (+ 1973us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:07.800142 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:07.800189 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:05:07.802409 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:07.802421 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:07.802885 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:07.803255 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:07.805962 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:07.807912 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:07.810658 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:07.811708 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:07.811896 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:07.817200 (+ 5304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:07.817207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:07.817219 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:07.817259 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:07.820021 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:07.820729 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:07.820738 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:07.820749 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:07.820891 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:07.821304 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:07.821309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:07.821313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:07.821762 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:07.821956 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:07.822627 (+ 671us) server_negotiation.cc:299] Negotiation successful
0806 01:05:07.822999 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1770,"thread_start_us":1586,"threads_started":1}
I20260806 01:05:07.823495 30938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:07.797786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:40097 (local address 127.5.108.129:60591)
0806 01:05:07.799728 (+ 1942us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:07.799752 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:07.799899 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:07.803487 (+ 3588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:07.803491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:07.804469 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:07.805718 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:07.805748 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:07.808127 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:07.808131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:07.810497 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:07.810511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:07.811972 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:07.811979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:07.812306 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:07.813229 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:07.813268 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:07.816915 (+ 3647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:07.820153 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:07.820159 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:07.820164 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:07.820582 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:07.821014 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:07.821019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:07.821022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:07.821171 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:07.821909 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:07.821914 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:07.822153 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 01:05:07.822406 (+ 253us) client_negotiation.cc:240] Negotiation successful
0806 01:05:07.823066 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1755,"mutex_wait_us":646,"spinlock_wait_cycles":26880,"thread_start_us":1502,"threads_started":1}
W20260806 01:05:07.826951 30358 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
W20260806 01:05:07.827207 30361 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
I20260806 01:05:07.827384 30361 leader_election.cc:302] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06aa81588cf24f76a8529c4b8d74823a; no voters: 04bc7c319ace448daa63d3c2ce33a93e, 298613e45bb2449eaeaca69b65abd237
I20260806 01:05:07.827867 30937 raft_consensus.cc:2726] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 01:05:08.142048 30942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:08.141288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:46927)
0806 01:05:08.141715 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:08.141840 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":207,"threads_started":1}
I20260806 01:05:08.605129 30944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:08.604437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:53197)
0806 01:05:08.604829 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:08.604950 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":194,"threads_started":1}
I20260806 01:05:08.664494 30946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:08.663784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:34453)
0806 01:05:08.664229 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:08.664303 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":227,"threads_started":1}
I20260806 01:05:09.143561 30947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:09.142879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:36823)
0806 01:05:09.143281 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:09.143353 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":208,"threads_started":1}
I20260806 01:05:09.561055 30948 raft_consensus.cc:482] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:09.561234 30948 raft_consensus.cc:504] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:05:09.561837 30948 leader_election.cc:288] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365), 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097)
W20260806 01:05:09.562948 30361 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
W20260806 01:05:09.563122 30358 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
I20260806 01:05:09.563266 30358 leader_election.cc:302] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06aa81588cf24f76a8529c4b8d74823a; no voters: 04bc7c319ace448daa63d3c2ce33a93e, 298613e45bb2449eaeaca69b65abd237
I20260806 01:05:09.563544 30948 raft_consensus.cc:2726] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 01:05:09.606801 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:09.605977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:43305)
0806 01:05:09.606536 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:09.606611 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":289,"threads_started":1}
I20260806 01:05:09.665956 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:09.665222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:55429)
0806 01:05:09.665703 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:09.665785 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":250,"threads_started":1}
I20260806 01:05:10.145054 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:10.144392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:56583)
0806 01:05:10.144834 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:10.144906 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":231,"threads_started":1}
I20260806 01:05:10.608274 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:10.607671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:47145)
0806 01:05:10.608061 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:10.608138 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":213,"threads_started":1}
I20260806 01:05:10.667426 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:10.666782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:33013)
0806 01:05:10.667171 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:10.667258 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":213,"threads_started":1}
I20260806 01:05:11.146776 30957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:11.146020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:38045)
0806 01:05:11.146497 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:11.146581 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
I20260806 01:05:11.610110 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:11.609398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:43281)
0806 01:05:11.609848 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:11.609936 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":205,"threads_started":1}
I20260806 01:05:11.669096 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:11.668366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:41569)
0806 01:05:11.668822 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:11.668916 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":211,"threads_started":1}
I20260806 01:05:12.148337 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:12.147649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:59077)
0806 01:05:12.148099 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:12.148179 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":218,"threads_started":1}
I20260806 01:05:12.278782 30965 raft_consensus.cc:482] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:12.279033 30965 raft_consensus.cc:504] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:05:12.279857 30965 leader_election.cc:288] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365), 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097)
W20260806 01:05:12.280835 30361 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
W20260806 01:05:12.281028 30358 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
I20260806 01:05:12.281211 30358 leader_election.cc:302] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06aa81588cf24f76a8529c4b8d74823a; no voters: 04bc7c319ace448daa63d3c2ce33a93e, 298613e45bb2449eaeaca69b65abd237
I20260806 01:05:12.281438 30965 raft_consensus.cc:2726] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 01:05:12.611707 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:12.611012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:40349)
0806 01:05:12.611461 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:12.611564 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":221,"threads_started":1}
I20260806 01:05:12.670686 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:12.670002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:56499)
0806 01:05:12.670424 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:12.670493 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":211,"threads_started":1}
I20260806 01:05:13.149868 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:13.149109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:56513)
0806 01:05:13.149615 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:13.149698 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":285,"threads_started":1}
I20260806 01:05:13.613201 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:13.612516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:34863)
0806 01:05:13.612958 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:13.613032 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":254,"threads_started":1}
I20260806 01:05:13.672288 30972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:13.671627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:47263)
0806 01:05:13.672037 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:13.672113 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":226,"threads_started":1}
I20260806 01:05:14.151633 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:14.150774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:54073)
0806 01:05:14.151284 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:14.151372 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":239,"threads_started":1}
I20260806 01:05:14.614975 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:14.614043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:35207)
0806 01:05:14.614629 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:14.614773 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":321,"threads_started":1}
I20260806 01:05:14.673681 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:14.673040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:49129)
0806 01:05:14.673472 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:14.673544 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":232,"threads_started":1}
I20260806 01:05:15.018186 30976 raft_consensus.cc:482] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:15.018419 30976 raft_consensus.cc:504] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:05:15.019191 30976 leader_election.cc:288] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365), 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097)
W20260806 01:05:15.020198 30358 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
W20260806 01:05:15.020395 30361 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
I20260806 01:05:15.020529 30361 leader_election.cc:302] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06aa81588cf24f76a8529c4b8d74823a; no voters: 04bc7c319ace448daa63d3c2ce33a93e, 298613e45bb2449eaeaca69b65abd237
I20260806 01:05:15.020828 30976 raft_consensus.cc:2726] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 01:05:15.153165 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:15.152471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:39747)
0806 01:05:15.152919 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:15.153004 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":264,"threads_started":1}
I20260806 01:05:15.616580 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:15.615866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:56897)
0806 01:05:15.616316 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:15.616403 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":246,"threads_started":1}
I20260806 01:05:15.675217 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:15.674563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:44799)
0806 01:05:15.674979 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:15.675051 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":225,"threads_started":1}
I20260806 01:05:16.154835 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:16.154133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:55689)
0806 01:05:16.154605 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:16.154675 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":223,"threads_started":1}
I20260806 01:05:16.618206 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:16.617526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:48447)
0806 01:05:16.617966 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:16.618054 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":208,"threads_started":1}
I20260806 01:05:16.676800 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:16.676104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:34575)
0806 01:05:16.676542 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:16.676627 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":224,"threads_started":1}
I20260806 01:05:17.156530 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:17.155770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:46269)
0806 01:05:17.156274 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:17.156351 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":211,"threads_started":1}
I20260806 01:05:17.619765 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:17.619036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:58673)
0806 01:05:17.619515 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:17.619591 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":244,"threads_started":1}
I20260806 01:05:17.678395 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:17.677691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:42441)
0806 01:05:17.678130 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:17.678214 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":243,"threads_started":1}
I20260806 01:05:18.158128 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:18.157386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:51737)
0806 01:05:18.157855 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:18.157948 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":238,"threads_started":1}
I20260806 01:05:18.621289 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:18.620586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:46207)
0806 01:05:18.621066 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:18.621139 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":255,"threads_started":1}
I20260806 01:05:18.680058 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:18.679357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:37353)
0806 01:05:18.679822 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:18.679904 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":200,"threads_started":1}
I20260806 01:05:19.159756 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:19.159001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:35269)
0806 01:05:19.159502 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:19.159584 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":245,"threads_started":1}
I20260806 01:05:19.622835 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:19.622140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:56959)
0806 01:05:19.622575 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:19.622663 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":239,"threads_started":1}
I20260806 01:05:19.681772 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:19.680871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:56773)
0806 01:05:19.681479 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:19.681574 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":390,"threads_started":1}
I20260806 01:05:20.161496 31002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:20.160681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:54899)
0806 01:05:20.161169 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:20.161312 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":269,"threads_started":1}
I20260806 01:05:20.624449 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:20.623754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:45253)
0806 01:05:20.624177 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:20.624271 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":225,"threads_started":1}
I20260806 01:05:20.683476 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:20.682689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:48721)
0806 01:05:20.683137 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:20.683215 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":240,"threads_started":1}
I20260806 01:05:21.163158 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:21.162438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:41941)
0806 01:05:21.162873 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:21.162967 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":237,"threads_started":1}
I20260806 01:05:21.625963 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:21.625252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:51783)
0806 01:05:21.625700 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:21.625774 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":226,"threads_started":1}
I20260806 01:05:21.685318 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:21.684641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.129:52775)
0806 01:05:21.685071 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:21.685146 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":217,"threads_started":1}
I20260806 01:05:22.141394 31010 raft_consensus.cc:482] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:22.141631 31010 raft_consensus.cc:504] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298613e45bb2449eaeaca69b65abd237" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 45365 } } peers { permanent_uuid: "04bc7c319ace448daa63d3c2ce33a93e" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 40097 } } peers { permanent_uuid: "06aa81588cf24f76a8529c4b8d74823a" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 39047 } }
I20260806 01:05:22.142356 31010 leader_election.cc:288] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365), 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097)
W20260806 01:05:22.143353 30361 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 298613e45bb2449eaeaca69b65abd237 (127.5.108.130:45365): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
W20260806 01:05:22.143612 30358 leader_election.cc:341] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04bc7c319ace448daa63d3c2ce33a93e (127.5.108.131:40097): Not found: Tablet not found: ae8ad49f8a0e4d5e92fcdb884b875593
I20260806 01:05:22.143707 30358 leader_election.cc:302] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06aa81588cf24f76a8529c4b8d74823a; no voters: 04bc7c319ace448daa63d3c2ce33a93e, 298613e45bb2449eaeaca69b65abd237
I20260806 01:05:22.143924 31010 raft_consensus.cc:2726] T ae8ad49f8a0e4d5e92fcdb884b875593 P 06aa81588cf24f76a8529c4b8d74823a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 01:05:22.165048 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:22.164244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.130:55183)
0806 01:05:22.164792 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:22.164880 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":342,"threads_started":1}
I20260806 01:05:22.617746 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 30269
I20260806 01:05:22.627780 31014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:22.626972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:34403 (local address 127.5.108.131:52561)
0806 01:05:22.627495 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:22.627592 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.190:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":275,"threads_started":1}
I20260806 01:05:22.637259 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:22.618875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.5.108.129:39047)
0806 01:05:22.619466 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:22.619476 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:22.619509 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:05:22.619588 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:22.619597 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:22.619858 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:22.620064 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:22.621873 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:22.622889 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:22.624283 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:22.624854 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:22.624953 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:22.632133 (+ 7180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:22.632145 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:22.632159 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:22.632188 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:22.634670 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:22.635261 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:22.635267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:22.635269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:22.635466 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:22.635800 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:22.635803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:22.635804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:22.636131 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:22.636374 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:22.636711 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 01:05:22.636940 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":497,"threads_started":1}
I20260806 01:05:22.638726 30512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:22.728463 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 30269
I20260806 01:05:22.757336 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 30529
I20260806 01:05:22.770560 31017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:22.758178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55958 (local address 127.5.108.130:45365)
0806 01:05:22.758708 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:22.758712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:22.758727 (+ 15us) server_negotiation.cc:407] Connection header received
0806 01:05:22.758794 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:22.758797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:22.758855 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:22.758943 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:22.760423 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:22.761178 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:22.762401 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:22.762807 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:22.762891 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:22.765591 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:22.765596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:22.765599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:22.765629 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:22.768301 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:22.768908 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:22.768911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:22.768914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:22.769033 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:22.769388 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:22.769391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:22.769393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:22.769621 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:22.769853 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:22.770092 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 01:05:22.770254 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":312,"threads_started":1}
I20260806 01:05:22.771250 30719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:22.853391 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 30529
I20260806 01:05:22.878238 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 30731
I20260806 01:05:22.891577 31020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:22.879180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.5.108.131:40097)
0806 01:05:22.879773 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:22.879778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:22.879794 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:05:22.879849 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:22.879853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:22.879918 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:22.880010 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:22.881353 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:22.882113 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:22.883328 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:22.883772 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:22.883863 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:22.886538 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:22.886543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:22.886546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:22.886575 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:22.889228 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:22.889864 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:22.889867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:22.889869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:22.889990 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:22.890318 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:22.890321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:22.890323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:22.890556 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:22.890756 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:22.891009 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 01:05:22.891159 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":342,"threads_started":1}
I20260806 01:05:22.892299 30923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:22.977051 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 30731
2026-08-06T01:05:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64655 ms)
[----------] 30 tests from SecurityITest (300156 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:23 dist-test-slave-x4qh krb5kdc[31030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:23 dist-test-slave-x4qh krb5kdc[31030](info): set up 2 sockets
Aug 06 01:05:23 dist-test-slave-x4qh krb5kdc[31030](info): commencing operation
krb5kdc: starting...
W20260806 01:05:24.264019 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.002s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:24 dist-test-slave-x4qh krb5kdc[31030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:05:24.541190 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36591
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:24.878561 31047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:24.878845 31047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:24.878932 31047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:24.885975 31047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:24.886076 31047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:24.886122 31047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:24.886180 31047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:24.886217 31047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:24.895582 31047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:24.896768 31047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:24.898015 31047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:24.911302 31052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:24.911325 31058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:24.911361 31053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:24.912350 31047 server_base.cc:1034] running on GCE node
I20260806 01:05:24.913144 31047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:24.914553 31047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:24.915802 31047 hybrid_clock.cc:648] HybridClock initialized: now 1785978324915773 us; error 54 us; skew 500 ppm
I20260806 01:05:24.916154 31047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:24 dist-test-slave-x4qh krb5kdc[31030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978324, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:24.919198 31047 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:05:24.920832 31047 webserver.cc:460] Webserver started at http://127.5.108.190:44463/ using document root <none> and password file <none>
I20260806 01:05:24.921521 31047 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:24.921679 31047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:24.922043 31047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:24.925092 31047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "de944ca69f8a4f70a6d400ce70375560"
format_stamp: "Formatted at 2026-08-06 01:05:24 on dist-test-slave-x4qh"
server_key: "1518b36031b05e3d34ec8258f4b48ce2"
server_key_iv: "e7b4816f81960dde20feecdcade9adfe"
server_key_version: "encryptionkey@0"
I20260806 01:05:24.925935 31047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "de944ca69f8a4f70a6d400ce70375560"
format_stamp: "Formatted at 2026-08-06 01:05:24 on dist-test-slave-x4qh"
server_key: "1518b36031b05e3d34ec8258f4b48ce2"
server_key_iv: "e7b4816f81960dde20feecdcade9adfe"
server_key_version: "encryptionkey@0"
I20260806 01:05:24.930944 31047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:05:24.934422 31071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:24.935673 31047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:24.935877 31047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "de944ca69f8a4f70a6d400ce70375560"
format_stamp: "Formatted at 2026-08-06 01:05:24 on dist-test-slave-x4qh"
server_key: "1518b36031b05e3d34ec8258f4b48ce2"
server_key_iv: "e7b4816f81960dde20feecdcade9adfe"
server_key_version: "encryptionkey@0"
I20260806 01:05:24.936084 31047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:24.946671 31047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:24.950555 31047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:24.950860 31047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:24.969892 31047 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36591
I20260806 01:05:24.969915 31148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36591 every 8 connection(s)
I20260806 01:05:24.971315 31047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:24.974900 31150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:24.975351 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 31047
I20260806 01:05:24.975610 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:24.976498 5554 external_mini_cluster.cc:1428] Setting key 3f32994a1b9a74171ec6a872de9ea6c8
I20260806 01:05:24.989125 31150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560: Bootstrap starting.
Aug 06 01:05:24 dist-test-slave-x4qh krb5kdc[31030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:24.992908 31150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:24.994310 31150 log.cc:824] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:24.997654 31150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560: No bootstrap required, opened a new log
I20260806 01:05:25.001147 31153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:24.978718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.5.108.190:36591)
0806 01:05:24.979507 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:24.979521 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:24.979561 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:05:24.981341 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:24.981366 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:24.981724 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:24.982040 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:24.983869 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:24.985144 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:24.987075 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:24.987800 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:24.987944 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:24.992715 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:24.992728 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:24.992747 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:24.992809 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:24.996674 (+ 3865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:24.997359 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:24.997367 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:24.997374 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:24.997625 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:24.998131 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:24.998136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:24.998139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:24.998794 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:24.999086 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:24.999452 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 01:05:24.999878 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":355,"threads_started":1}
I20260806 01:05:25.006561 31150 raft_consensus.cc:348] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de944ca69f8a4f70a6d400ce70375560" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36591 } }
I20260806 01:05:25.006882 31150 raft_consensus.cc:374] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:25.006966 31150 raft_consensus.cc:729] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de944ca69f8a4f70a6d400ce70375560, State: Initialized, Role: FOLLOWER
I20260806 01:05:25.007498 31150 consensus_queue.cc:260] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de944ca69f8a4f70a6d400ce70375560" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36591 } }
I20260806 01:05:25.007709 31150 raft_consensus.cc:388] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:25.007808 31150 raft_consensus.cc:482] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:25.007923 31150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:25.010136 31150 raft_consensus.cc:504] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de944ca69f8a4f70a6d400ce70375560" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36591 } }
I20260806 01:05:25.010556 31150 leader_election.cc:302] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de944ca69f8a4f70a6d400ce70375560; no voters:
I20260806 01:05:25.011049 31150 leader_election.cc:288] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:25.011153 31156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:25.011921 31156 raft_consensus.cc:686] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [term 1 LEADER]: Becoming Leader. State: Replica: de944ca69f8a4f70a6d400ce70375560, State: Running, Role: LEADER
I20260806 01:05:25.012465 31156 consensus_queue.cc:237] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de944ca69f8a4f70a6d400ce70375560" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36591 } }
I20260806 01:05:25.013005 31150 sys_catalog.cc:564] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:25.016057 31162 sys_catalog.cc:455] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de944ca69f8a4f70a6d400ce70375560. Latest consensus state: current_term: 1 leader_uuid: "de944ca69f8a4f70a6d400ce70375560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de944ca69f8a4f70a6d400ce70375560" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36591 } } }
I20260806 01:05:25.016945 31162 sys_catalog.cc:458] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:25.016085 31158 sys_catalog.cc:455] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de944ca69f8a4f70a6d400ce70375560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de944ca69f8a4f70a6d400ce70375560" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36591 } } }
I20260806 01:05:25.017144 31158 sys_catalog.cc:458] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:25.020848 31170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:25.025830 31170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:25.034449 31170 catalog_manager.cc:1347] Generated new cluster ID: 38a37b459a704e4fbface948d1c766ff
I20260806 01:05:25.034547 31170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:25.046561 31170 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:25.047894 31170 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:25.053961 31170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560: Generated new TSK 0
I20260806 01:05:25.054754 31170 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:05:25.129420 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36591
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:25.482606 31188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:25.482884 31188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:25.483074 31188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:25.490464 31188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:25.490569 31188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:25.490803 31188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:05:25.500849 31188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:25.502122 31188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:25.503515 31188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:25.519425 31195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:25.519395 31198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:25.519721 31193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:25.520385 31188 server_base.cc:1034] running on GCE node
I20260806 01:05:25.521059 31188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:25.521951 31188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:25.523185 31188 hybrid_clock.cc:648] HybridClock initialized: now 1785978325523131 us; error 50 us; skew 500 ppm
I20260806 01:05:25.523547 31188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:25 dist-test-slave-x4qh krb5kdc[31030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978325, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:25.526688 31188 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:05:25.528127 31188 webserver.cc:460] Webserver started at http://127.5.108.129:42055/ using document root <none> and password file <none>
I20260806 01:05:25.528748 31188 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:25.528851 31188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:25.529165 31188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:25.532006 31188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "37fd96d38f5141ce91ca6130bcb09537"
format_stamp: "Formatted at 2026-08-06 01:05:25 on dist-test-slave-x4qh"
server_key: "0bbdb6e9f4b3fc6ac60be8d086bf00f1"
server_key_iv: "edfc60be5f4ac490910d2c4745c8ccce"
server_key_version: "encryptionkey@0"
I20260806 01:05:25.532792 31188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "37fd96d38f5141ce91ca6130bcb09537"
format_stamp: "Formatted at 2026-08-06 01:05:25 on dist-test-slave-x4qh"
server_key: "0bbdb6e9f4b3fc6ac60be8d086bf00f1"
server_key_iv: "edfc60be5f4ac490910d2c4745c8ccce"
server_key_version: "encryptionkey@0"
I20260806 01:05:25.537648 31188 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 01:05:25.541162 31205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:25.542342 31188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:05:25.542518 31188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "37fd96d38f5141ce91ca6130bcb09537"
format_stamp: "Formatted at 2026-08-06 01:05:25 on dist-test-slave-x4qh"
server_key: "0bbdb6e9f4b3fc6ac60be8d086bf00f1"
server_key_iv: "edfc60be5f4ac490910d2c4745c8ccce"
server_key_version: "encryptionkey@0"
I20260806 01:05:25.542712 31188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:25.564021 31188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:25.567957 31188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:25.568256 31188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:25.569208 31188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:25.570648 31188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:25.570758 31188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:25.570919 31188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:25.570994 31188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:25.604696 31188 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:38125
I20260806 01:05:25.604779 31375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:38125 every 8 connection(s)
I20260806 01:05:25.605960 31188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:05:25.610889 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 31188
I20260806 01:05:25.611124 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:05:25.611872 5554 external_mini_cluster.cc:1428] Setting key 21979cc3de99d640ec21c2faac952adb
Aug 06 01:05:25 dist-test-slave-x4qh krb5kdc[31030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978325, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:25.628139 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:25.608570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:35305 (local address 127.5.108.190:36591)
0806 01:05:25.609063 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:25.609067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:25.609805 (+ 738us) server_negotiation.cc:407] Connection header received
0806 01:05:25.611799 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:25.611805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:25.611899 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:25.612036 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:25.613922 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:25.615060 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:25.616678 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:25.617417 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:25.617564 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:25.620969 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:25.620978 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:25.620985 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:25.621030 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:25.624617 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:25.625439 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:25.625447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:25.625453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:25.625606 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:25.626072 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:25.626079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:25.626085 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:25.626464 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:25.626619 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:25.627415 (+ 796us) server_negotiation.cc:299] Negotiation successful
0806 01:05:25.627664 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":272,"threads_started":1}
I20260806 01:05:25.628937 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:25.608924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36591 (local address 127.5.108.129:35305)
0806 01:05:25.609619 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:25.609663 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:25.611520 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:25.612270 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:25.612278 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:25.612924 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:25.613703 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:25.613762 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:25.615257 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:25.615261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:25.616526 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:25.616538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:25.617637 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:25.617646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:25.617857 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:25.618529 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:25.618550 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:25.620767 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:25.624751 (+ 3984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:25.624757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:25.624767 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:25.625303 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:25.625716 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:25.625723 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:25.625729 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:25.625949 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:25.626589 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:25.626598 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:25.626978 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 01:05:25.627228 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 01:05:25.627719 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":287,"threads_started":1}
I20260806 01:05:25.630470 31380 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36591
I20260806 01:05:25.630776 31380 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:25.631489 31380 heartbeater.cc:511] Master 127.5.108.190:36591 requested a full tablet report, sending...
I20260806 01:05:25.633726 31094 ts_manager.cc:194] Registered new tserver with Master: 37fd96d38f5141ce91ca6130bcb09537 (127.5.108.129:38125)
I20260806 01:05:25.635898 31094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:35305
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:05:25.683212 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36591
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:26.016458 31391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:26.016775 31391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:26.016966 31391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:26.024283 31391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:26.024448 31391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:26.024628 31391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:05:26.034080 31391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:26.035362 31391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:26.036594 31391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:26.049993 31406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:26.050027 31408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:26.050209 31411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:26.050709 31391 server_base.cc:1034] running on GCE node
I20260806 01:05:26.051288 31391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:26.052232 31391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:26.053467 31391 hybrid_clock.cc:648] HybridClock initialized: now 1785978326053425 us; error 54 us; skew 500 ppm
I20260806 01:05:26.053777 31391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:26 dist-test-slave-x4qh krb5kdc[31030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978326, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:26.057061 31391 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:05:26.058501 31391 webserver.cc:460] Webserver started at http://127.5.108.130:45123/ using document root <none> and password file <none>
I20260806 01:05:26.059145 31391 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:26.059293 31391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:26.059633 31391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:26.062413 31391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "a8583f027754413296cd67765a102d43"
format_stamp: "Formatted at 2026-08-06 01:05:26 on dist-test-slave-x4qh"
server_key: "c02de078e8d5a6ed9977117bff12a713"
server_key_iv: "8dbbe1e50ce5dfcb53164a438766d98a"
server_key_version: "encryptionkey@0"
I20260806 01:05:26.063184 31391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "a8583f027754413296cd67765a102d43"
format_stamp: "Formatted at 2026-08-06 01:05:26 on dist-test-slave-x4qh"
server_key: "c02de078e8d5a6ed9977117bff12a713"
server_key_iv: "8dbbe1e50ce5dfcb53164a438766d98a"
server_key_version: "encryptionkey@0"
I20260806 01:05:26.068287 31391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:05:26.071512 31418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:26.072674 31391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:26.072847 31391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "a8583f027754413296cd67765a102d43"
format_stamp: "Formatted at 2026-08-06 01:05:26 on dist-test-slave-x4qh"
server_key: "c02de078e8d5a6ed9977117bff12a713"
server_key_iv: "8dbbe1e50ce5dfcb53164a438766d98a"
server_key_version: "encryptionkey@0"
I20260806 01:05:26.073028 31391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:26.092227 31391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:26.096413 31391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:26.096714 31391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:26.097631 31391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:26.098894 31391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:26.098992 31391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:26.099135 31391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:26.099205 31391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:26.132941 31391 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:33499
I20260806 01:05:26.133021 31583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:33499 every 8 connection(s)
I20260806 01:05:26.134155 31391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:05:26.139705 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 31391
I20260806 01:05:26.139900 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:05:26.140568 5554 external_mini_cluster.cc:1428] Setting key ea07ca52c2ff8cc7b35d3b51d5388d39
Aug 06 01:05:26 dist-test-slave-x4qh krb5kdc[31030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978326, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:26.154294 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.136604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:34947 (local address 127.5.108.190:36591)
0806 01:05:26.137040 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.137044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.138024 (+ 980us) server_negotiation.cc:407] Connection header received
0806 01:05:26.140198 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.140205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.140286 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.140473 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:26.142183 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.142899 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.144454 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.144879 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.145010 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.148619 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:26.148624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:26.148627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:26.148659 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:26.151255 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:26.151986 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:26.151991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:26.151994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:26.152143 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:26.152589 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:26.152593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:26.152595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:26.152876 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:26.153035 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.153727 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.153952 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":222,"threads_started":1}
I20260806 01:05:26.155158 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.137155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36591 (local address 127.5.108.130:34947)
0806 01:05:26.137866 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:26.137905 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:26.139888 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:26.140750 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:26.140761 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:26.141437 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:26.141997 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.142015 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.143040 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.143049 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.144264 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.144277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.145118 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.145126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.145353 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.146006 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:26.146031 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:26.148388 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:26.151419 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:26.151427 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:26.151442 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:26.151852 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:26.152263 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:26.152267 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:26.152271 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:26.152457 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:26.153010 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:26.153017 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:26.153351 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 01:05:26.153613 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 01:05:26.154103 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":280,"threads_started":1}
I20260806 01:05:26.156574 31584 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36591
I20260806 01:05:26.156903 31584 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:26.157634 31584 heartbeater.cc:511] Master 127.5.108.190:36591 requested a full tablet report, sending...
I20260806 01:05:26.158972 31094 ts_manager.cc:194] Registered new tserver with Master: a8583f027754413296cd67765a102d43 (127.5.108.130:33499)
I20260806 01:05:26.159852 31094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:34947
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:05:26.204481 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36591
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:26.536123 31596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:26.536360 31596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:26.536554 31596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:26.543669 31596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:26.543792 31596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:26.543931 31596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:05:26.552948 31596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:36591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:26.554013 31596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:26.555135 31596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:26.567998 31609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:26.568113 31613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:26.568086 31606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:26.568825 31596 server_base.cc:1034] running on GCE node
I20260806 01:05:26.569393 31596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:26.570303 31596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:26.571514 31596 hybrid_clock.cc:648] HybridClock initialized: now 1785978326571474 us; error 49 us; skew 500 ppm
I20260806 01:05:26.571851 31596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:26 dist-test-slave-x4qh krb5kdc[31030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978326, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:26.574887 31596 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:05:26.576382 31596 webserver.cc:460] Webserver started at http://127.5.108.131:42283/ using document root <none> and password file <none>
I20260806 01:05:26.577139 31596 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:26.577292 31596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:26.577605 31596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:26.580413 31596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "1743b51229ef4efe981bb6281e2f8499"
format_stamp: "Formatted at 2026-08-06 01:05:26 on dist-test-slave-x4qh"
server_key: "f538157ab34c6607a50e99a1b2970550"
server_key_iv: "9646bec4eed38b0c9a5ad4f3ee87e457"
server_key_version: "encryptionkey@0"
I20260806 01:05:26.581175 31596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "1743b51229ef4efe981bb6281e2f8499"
format_stamp: "Formatted at 2026-08-06 01:05:26 on dist-test-slave-x4qh"
server_key: "f538157ab34c6607a50e99a1b2970550"
server_key_iv: "9646bec4eed38b0c9a5ad4f3ee87e457"
server_key_version: "encryptionkey@0"
I20260806 01:05:26.586040 31596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:05:26.589450 31623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:26.590668 31596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 01:05:26.590831 31596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "1743b51229ef4efe981bb6281e2f8499"
format_stamp: "Formatted at 2026-08-06 01:05:26 on dist-test-slave-x4qh"
server_key: "f538157ab34c6607a50e99a1b2970550"
server_key_iv: "9646bec4eed38b0c9a5ad4f3ee87e457"
server_key_version: "encryptionkey@0"
I20260806 01:05:26.591030 31596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:26.630892 31596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:26.634505 31596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:26.634799 31596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:26.635736 31596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:26.637041 31596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:26.637145 31596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:26.637296 31596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:26.637372 31596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:26.638674 31380 heartbeater.cc:503] Master 127.5.108.190:36591 was elected leader, sending a full tablet report...
I20260806 01:05:26.672691 31596 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:34491
I20260806 01:05:26.672714 31776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:34491 every 8 connection(s)
I20260806 01:05:26.674008 31596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:05:26.682703 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 31596
I20260806 01:05:26.682901 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:05:26.683322 5554 external_mini_cluster.cc:1428] Setting key df123f5099664c2d8f24b38b98bd2f7a
Aug 06 01:05:26 dist-test-slave-x4qh krb5kdc[31030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978326, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:26.693751 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.676702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:43331 (local address 127.5.108.190:36591)
0806 01:05:26.677220 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.677226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.678245 (+ 1019us) server_negotiation.cc:407] Connection header received
0806 01:05:26.680094 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.680100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.680195 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.680312 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:26.681750 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.682544 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.684396 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.684962 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.685064 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.688744 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:26.688749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:26.688752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:26.688784 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:26.691130 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:26.691875 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:26.691879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:26.691881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:26.692000 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:26.692365 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:26.692368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:26.692370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:26.692610 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:26.692743 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.693312 (+ 569us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.693466 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":224,"threads_started":1}
I20260806 01:05:26.694679 31785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.677371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:36591 (local address 127.5.108.131:43331)
0806 01:05:26.678066 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:26.678117 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:26.679828 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:26.680540 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:26.680548 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:26.681048 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:26.681588 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.681603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.682697 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.682709 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.684131 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.684152 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.685199 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.685208 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.685467 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.686263 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:26.686299 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:26.688564 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:26.691246 (+ 2682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:26.691252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:26.691263 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:26.691766 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:26.692090 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:26.692093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:26.692096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:26.692268 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:26.692723 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:26.692728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:26.693002 (+ 274us) client_negotiation.cc:769] Sending connection context
0806 01:05:26.693213 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 01:05:26.693550 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":302,"threads_started":1}
I20260806 01:05:26.696134 31778 heartbeater.cc:347] Connected to a master server at 127.5.108.190:36591
I20260806 01:05:26.696405 31778 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:26.697160 31778 heartbeater.cc:511] Master 127.5.108.190:36591 requested a full tablet report, sending...
I20260806 01:05:26.698755 31094 ts_manager.cc:194] Registered new tserver with Master: 1743b51229ef4efe981bb6281e2f8499 (127.5.108.131:34491)
I20260806 01:05:26.699607 31094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:43331
I20260806 01:05:26.710459 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:05:26.727911 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.715293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.5.108.190:36591)
0806 01:05:26.715516 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.715524 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.715900 (+ 376us) server_negotiation.cc:407] Connection header received
0806 01:05:26.716106 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.716111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.716178 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.716285 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:26.717775 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.718515 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.719875 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.720301 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.720434 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.722631 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:26.722637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:26.722643 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:26.722677 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:26.725103 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:26.725871 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:26.725878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:26.725882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:26.726002 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:26.726403 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:26.726411 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:26.726414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:26.726764 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:26.727027 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.727343 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.727596 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:05:26.731655 31094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:05:26.738404 31094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:05:26.762765 31805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.750948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.5.108.129:38125)
0806 01:05:26.752037 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.752043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.752068 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:05:26.752474 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.752481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.752747 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.752946 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:26.753795 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.755411 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.758248 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.759708 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.759847 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.760076 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.762241 (+ 2165us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.762440 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932,"thread_start_us":788,"threads_started":1}
I20260806 01:05:26.761310 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.750793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38125 (local address 127.0.0.1:49324)
0806 01:05:26.751890 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:26.751934 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:26.752104 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:26.752984 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:26.752993 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:26.753029 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:26.753539 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.753555 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.755586 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.755593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.758038 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.758056 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.760216 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.760222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.760407 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.760452 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 01:05:26.760585 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 01:05:26.760776 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"spinlock_wait_cycles":13568,"thread_start_us":1648,"threads_started":1}
I20260806 01:05:26.764823 31807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.752296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:34491 (local address 127.0.0.1:39224)
0806 01:05:26.752939 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:26.752962 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:26.753278 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:26.755294 (+ 2016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:26.755301 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:26.755332 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:26.755946 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.755959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.759049 (+ 3090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.759054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.761103 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.761125 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.764021 (+ 2896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.764027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.764218 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.764273 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 01:05:26.764364 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 01:05:26.764495 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":1059,"threads_started":1}
I20260806 01:05:26.766042 31806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.751917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39224 (local address 127.5.108.131:34491)
0806 01:05:26.752665 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.752672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.754541 (+ 1869us) server_negotiation.cc:407] Connection header received
0806 01:05:26.754653 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.754659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.754855 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.755050 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:26.756197 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.758898 (+ 2701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.762380 (+ 3482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.763792 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.765213 (+ 1421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.765447 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.765514 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.765678 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":428,"threads_started":1}
I20260806 01:05:26.767225 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.753197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:33499 (local address 127.0.0.1:42040)
0806 01:05:26.753916 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:26.753944 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:26.754076 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:26.755355 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:26.755359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:26.755407 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:26.756417 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.756436 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.760516 (+ 4080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.760560 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.763581 (+ 3021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.763601 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.766229 (+ 2628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.766236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.766435 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.766491 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 01:05:26.766572 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 01:05:26.766755 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":1391,"threads_started":1}
I20260806 01:05:26.767249 31281 tablet_service.cc:1467] Processing CreateTablet for tablet 253dd1af5c734bcb920a8953a72c3d9e (DEFAULT_TABLE table=test-table [id=189fb6e7a4a2453e894807374cbc82b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:26.768349 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.753417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.5.108.130:33499)
0806 01:05:26.754579 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.754588 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.754608 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:05:26.754699 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.754705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.754873 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.755050 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:26.757013 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.760305 (+ 3292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.763837 (+ 3532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.765942 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.767599 (+ 1657us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.767787 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.767855 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.768017 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1007,"thread_start_us":898,"threads_started":1}
I20260806 01:05:26.768862 31281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 253dd1af5c734bcb920a8953a72c3d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:26.770421 31688 tablet_service.cc:1467] Processing CreateTablet for tablet 253dd1af5c734bcb920a8953a72c3d9e (DEFAULT_TABLE table=test-table [id=189fb6e7a4a2453e894807374cbc82b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:26.771632 31688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 253dd1af5c734bcb920a8953a72c3d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:26.771785 31492 tablet_service.cc:1467] Processing CreateTablet for tablet 253dd1af5c734bcb920a8953a72c3d9e (DEFAULT_TABLE table=test-table [id=189fb6e7a4a2453e894807374cbc82b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:26.773083 31492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 253dd1af5c734bcb920a8953a72c3d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:26.783450 31811 tablet_bootstrap.cc:492] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: Bootstrap starting.
I20260806 01:05:26.785449 31812 tablet_bootstrap.cc:492] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537: Bootstrap starting.
I20260806 01:05:26.787672 31811 tablet_bootstrap.cc:654] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:26.788967 31812 tablet_bootstrap.cc:654] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:26.789409 31811 log.cc:824] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:26.790180 31812 log.cc:824] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:26.792088 31813 tablet_bootstrap.cc:492] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: Bootstrap starting.
I20260806 01:05:26.794796 31812 tablet_bootstrap.cc:492] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537: No bootstrap required, opened a new log
I20260806 01:05:26.795055 31812 ts_tablet_manager.cc:1397] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260806 01:05:26.795483 31811 tablet_bootstrap.cc:492] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: No bootstrap required, opened a new log
I20260806 01:05:26.795485 31813 tablet_bootstrap.cc:654] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:26.795755 31811 ts_tablet_manager.cc:1397] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260806 01:05:26.796703 31813 log.cc:824] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:26.799702 31813 tablet_bootstrap.cc:492] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: No bootstrap required, opened a new log
I20260806 01:05:26.799960 31813 ts_tablet_manager.cc:1397] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 01:05:26.802974 31812 raft_consensus.cc:348] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } }
I20260806 01:05:26.803342 31812 raft_consensus.cc:374] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:26.803489 31812 raft_consensus.cc:729] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37fd96d38f5141ce91ca6130bcb09537, State: Initialized, Role: FOLLOWER
I20260806 01:05:26.804081 31812 consensus_queue.cc:260] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } }
I20260806 01:05:26.805856 31812 ts_tablet_manager.cc:1428] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260806 01:05:26.805994 31811 raft_consensus.cc:348] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } }
I20260806 01:05:26.806413 31811 raft_consensus.cc:374] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:26.806512 31811 raft_consensus.cc:729] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1743b51229ef4efe981bb6281e2f8499, State: Initialized, Role: FOLLOWER
I20260806 01:05:26.807081 31811 consensus_queue.cc:260] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } }
I20260806 01:05:26.808281 31778 heartbeater.cc:503] Master 127.5.108.190:36591 was elected leader, sending a full tablet report...
I20260806 01:05:26.808728 31811 ts_tablet_manager.cc:1428] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 01:05:26.810104 31813 raft_consensus.cc:348] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } }
I20260806 01:05:26.810441 31813 raft_consensus.cc:374] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:26.810541 31813 raft_consensus.cc:729] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8583f027754413296cd67765a102d43, State: Initialized, Role: FOLLOWER
I20260806 01:05:26.811192 31813 consensus_queue.cc:260] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } }
I20260806 01:05:26.812703 31813 ts_tablet_manager.cc:1428] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 01:05:26.813001 31584 heartbeater.cc:503] Master 127.5.108.190:36591 was elected leader, sending a full tablet report...
I20260806 01:05:26.818326 31822 raft_consensus.cc:482] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:26.818555 31822 raft_consensus.cc:504] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } }
I20260806 01:05:26.819989 31822 leader_election.cc:288] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37fd96d38f5141ce91ca6130bcb09537 (127.5.108.129:38125), a8583f027754413296cd67765a102d43 (127.5.108.130:33499)
I20260806 01:05:26.826336 31805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.820339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:53533 (local address 127.5.108.129:38125)
0806 01:05:26.820571 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.820577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.820596 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:05:26.820813 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.820816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.820893 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.820970 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:26.821461 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.822472 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.824354 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.825574 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.825748 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.825807 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.826089 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.826171 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:05:26.826467 31785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.820323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38125 (local address 127.5.108.131:53533)
0806 01:05:26.820504 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:26.820519 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:26.820645 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:26.821132 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:26.821135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:26.821156 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:26.821310 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.821316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.822588 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.822594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.824191 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.824204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.825737 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.825744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.825931 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.825965 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:05:26.826062 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 01:05:26.826203 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":41}
I20260806 01:05:26.827448 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.820815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:59645 (local address 127.5.108.130:33499)
0806 01:05:26.821017 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.821022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.821454 (+ 432us) server_negotiation.cc:407] Connection header received
0806 01:05:26.821599 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.821602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.821650 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.821727 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:26.822321 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.823270 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.825395 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.826561 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.826725 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.826792 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.827071 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.827189 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:05:26.827601 31824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.820672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:33499 (local address 127.5.108.131:59645)
0806 01:05:26.821314 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:26.821327 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:26.821478 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:26.821777 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:26.821782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:26.821806 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:26.822172 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.822183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.823465 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.823468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.825213 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:26.825229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.826751 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.826757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:26.826901 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.826922 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 01:05:26.827036 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 01:05:26.827155 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":3,"thread_start_us":355,"threads_started":1}
I20260806 01:05:26.827472 31308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "253dd1af5c734bcb920a8953a72c3d9e" candidate_uuid: "1743b51229ef4efe981bb6281e2f8499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fd96d38f5141ce91ca6130bcb09537" is_pre_election: true
I20260806 01:05:26.828035 31308 raft_consensus.cc:2454] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1743b51229ef4efe981bb6281e2f8499 in term 0.
I20260806 01:05:26.828598 31520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "253dd1af5c734bcb920a8953a72c3d9e" candidate_uuid: "1743b51229ef4efe981bb6281e2f8499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8583f027754413296cd67765a102d43" is_pre_election: true
I20260806 01:05:26.829092 31520 raft_consensus.cc:2454] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1743b51229ef4efe981bb6281e2f8499 in term 0.
I20260806 01:05:26.829706 31627 leader_election.cc:302] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1743b51229ef4efe981bb6281e2f8499, 37fd96d38f5141ce91ca6130bcb09537; no voters:
I20260806 01:05:26.830015 31822 raft_consensus.cc:2781] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:05:26.830158 31822 raft_consensus.cc:482] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:05:26.830281 31822 raft_consensus.cc:3037] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:26.832444 31822 raft_consensus.cc:504] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } }
I20260806 01:05:26.833006 31822 leader_election.cc:288] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [CANDIDATE]: Term 1 election: Requested vote from peers 37fd96d38f5141ce91ca6130bcb09537 (127.5.108.129:38125), a8583f027754413296cd67765a102d43 (127.5.108.130:33499)
I20260806 01:05:26.833350 31308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "253dd1af5c734bcb920a8953a72c3d9e" candidate_uuid: "1743b51229ef4efe981bb6281e2f8499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fd96d38f5141ce91ca6130bcb09537"
I20260806 01:05:26.833423 31520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "253dd1af5c734bcb920a8953a72c3d9e" candidate_uuid: "1743b51229ef4efe981bb6281e2f8499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8583f027754413296cd67765a102d43"
I20260806 01:05:26.833524 31308 raft_consensus.cc:3037] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:26.833649 31520 raft_consensus.cc:3037] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:26.835724 31308 raft_consensus.cc:2454] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1743b51229ef4efe981bb6281e2f8499 in term 1.
I20260806 01:05:26.835945 31520 raft_consensus.cc:2454] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1743b51229ef4efe981bb6281e2f8499 in term 1.
I20260806 01:05:26.836131 31627 leader_election.cc:302] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1743b51229ef4efe981bb6281e2f8499, 37fd96d38f5141ce91ca6130bcb09537; no voters:
I20260806 01:05:26.836391 31822 raft_consensus.cc:2781] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:26.836829 31822 raft_consensus.cc:686] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 1 LEADER]: Becoming Leader. State: Replica: 1743b51229ef4efe981bb6281e2f8499, State: Running, Role: LEADER
I20260806 01:05:26.837256 31822 consensus_queue.cc:237] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } }
I20260806 01:05:26.841747 31094 catalog_manager.cc:5495] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1743b51229ef4efe981bb6281e2f8499 (127.5.108.131). New cstate: current_term: 1 leader_uuid: "1743b51229ef4efe981bb6281e2f8499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1743b51229ef4efe981bb6281e2f8499" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34491 } health_report { overall_health: HEALTHY } } }
W20260806 01:05:26.858387 31381 tablet.cc:2369] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:05:26.859928 31806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.854479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.5.108.131:34491)
0806 01:05:26.854684 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.854690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.854811 (+ 121us) server_negotiation.cc:407] Connection header received
0806 01:05:26.854985 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.854988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.855055 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.855161 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:05:26.855542 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.856203 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.857669 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.858104 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.858249 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.858762 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:05:26.859413 (+ 651us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:05:26.859525 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.859618 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.859681 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:05:26.867519 31308 raft_consensus.cc:1261] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [term 1 FOLLOWER]: Refusing update from remote peer 1743b51229ef4efe981bb6281e2f8499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:26.867517 31520 raft_consensus.cc:1261] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [term 1 FOLLOWER]: Refusing update from remote peer 1743b51229ef4efe981bb6281e2f8499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:26.868392 31822 consensus_queue.cc:1035] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37fd96d38f5141ce91ca6130bcb09537" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:26.868708 31826 consensus_queue.cc:1035] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8583f027754413296cd67765a102d43" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:26.894302 31094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:26.894840 31094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:26.899617 31094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de944ca69f8a4f70a6d400ce70375560: Sending DeleteTablet for 3 replicas of tablet 253dd1af5c734bcb920a8953a72c3d9e
I20260806 01:05:26.900790 31492 tablet_service.cc:1514] Processing DeleteTablet for tablet 253dd1af5c734bcb920a8953a72c3d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:26 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:42040
I20260806 01:05:26.900835 31281 tablet_service.cc:1514] Processing DeleteTablet for tablet 253dd1af5c734bcb920a8953a72c3d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:26 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:49324
I20260806 01:05:26.901755 31688 tablet_service.cc:1514] Processing DeleteTablet for tablet 253dd1af5c734bcb920a8953a72c3d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:26 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:39224
I20260806 01:05:26.902307 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 31188
Aug 06 01:05:26 dist-test-slave-x4qh krb5kdc[31030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:05:26.928166 31848 tablet_replica.cc:330] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: stopping tablet replica
I20260806 01:05:26.928682 31848 raft_consensus.cc:2229] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:05:26.929425 31848 raft_consensus.cc:2258] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:26.930263 31846 tablet_replica.cc:330] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537: stopping tablet replica
I20260806 01:05:26.930660 31846 raft_consensus.cc:2229] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:26.931124 31846 raft_consensus.cc:2258] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:26.933347 31848 ts_tablet_manager.cc:1905] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:26.935997 31847 tablet_replica.cc:330] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: stopping tablet replica
I20260806 01:05:26.936436 31847 raft_consensus.cc:2229] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:26.936932 31847 raft_consensus.cc:2258] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:26.937263 31846 ts_tablet_manager.cc:1905] T 253dd1af5c734bcb920a8953a72c3d9e P 37fd96d38f5141ce91ca6130bcb09537: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:26.938197 31805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:26.903973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.5.108.129:38125)
0806 01:05:26.904186 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:26.904194 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:26.914465 (+ 10271us) server_negotiation.cc:407] Connection header received
0806 01:05:26.915002 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:26.915012 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:26.915122 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:26.915208 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:26.917540 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.919106 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.921585 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:26.922399 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:26.922512 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:26.929661 (+ 7149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:26.929678 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:26.929699 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:26.929763 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:26.934607 (+ 4844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:26.935671 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:26.935678 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:26.935681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:26.935965 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:26.936482 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:26.936486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:26.936488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:26.936901 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:26.937258 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:26.937652 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 01:05:26.937919 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:05:26.939666 31369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:26.940024 31847 ts_tablet_manager.cc:1905] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:26.946008 31847 ts_tablet_manager.cc:1918] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:26.946169 31847 log.cc:1196] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/253dd1af5c734bcb920a8953a72c3d9e
I20260806 01:05:26.946779 31847 ts_tablet_manager.cc:1939] T 253dd1af5c734bcb920a8953a72c3d9e P a8583f027754413296cd67765a102d43: Deleting consensus metadata
I20260806 01:05:26.948623 31075 catalog_manager.cc:4841] TS a8583f027754413296cd67765a102d43 (127.5.108.130:33499): tablet 253dd1af5c734bcb920a8953a72c3d9e (table test-table [id=189fb6e7a4a2453e894807374cbc82b8]) successfully deleted
I20260806 01:05:26.951315 31848 ts_tablet_manager.cc:1918] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:26.951506 31848 log.cc:1196] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/253dd1af5c734bcb920a8953a72c3d9e
I20260806 01:05:26.951967 31848 ts_tablet_manager.cc:1939] T 253dd1af5c734bcb920a8953a72c3d9e P 1743b51229ef4efe981bb6281e2f8499: Deleting consensus metadata
I20260806 01:05:26.953383 31072 catalog_manager.cc:4841] TS 1743b51229ef4efe981bb6281e2f8499 (127.5.108.131:34491): tablet 253dd1af5c734bcb920a8953a72c3d9e (table test-table [id=189fb6e7a4a2453e894807374cbc82b8]) successfully deleted
I20260806 01:05:27.043303 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 31188
I20260806 01:05:27.068585 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 31391
W20260806 01:05:27.068728 31075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.108.129:38125 (error 108)
I20260806 01:05:27.072494 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:27.071971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38125 (local address 127.0.0.1:49336)
0806 01:05:27.072181 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:27.072321 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 01:05:27.072784 31075 catalog_manager.cc:4584] TS 37fd96d38f5141ce91ca6130bcb09537 (127.5.108.129:38125): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 253dd1af5c734bcb920a8953a72c3d9e: Network error: Client connection negotiation failed: client connection to 127.5.108.129:38125: connect: Connection refused (error 111)
Aug 06 01:05:27 dist-test-slave-x4qh krb5kdc[31030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:05:27.085099 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:27.069547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42046 (local address 127.5.108.130:33499)
0806 01:05:27.069748 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:27.069755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:27.070144 (+ 389us) server_negotiation.cc:407] Connection header received
0806 01:05:27.070231 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:27.070235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:27.070299 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:27.070426 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:27.071748 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:27.072632 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:27.073999 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:27.074385 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:27.074520 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:27.079293 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:27.079308 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:27.079322 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:27.079356 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:27.082274 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:27.082974 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:27.082991 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:27.082993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:27.083118 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:27.083568 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:27.083573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:27.083576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:27.083947 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:27.084253 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:27.084557 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 01:05:27.084771 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 01:05:27.085712 31578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:27.126595 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:27.125898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38125 (local address 127.0.0.1:49338)
0806 01:05:27.126294 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:27.126412 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233}
I20260806 01:05:27.127924 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:27.127450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38125 (local address 127.0.0.1:49340)
0806 01:05:27.127685 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:27.127769 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260806 01:05:27.173964 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 31391
I20260806 01:05:27.188557 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:27.188038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38125 (local address 127.0.0.1:49354)
0806 01:05:27.188303 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:27.188396 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 01:05:27.189615 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:27.189276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38125 (local address 127.0.0.1:49358)
0806 01:05:27.189423 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:27.189501 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 01:05:27.205353 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 31596
Aug 06 01:05:27 dist-test-slave-x4qh krb5kdc[31030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:05:27.223008 31806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:27.206162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39236 (local address 127.5.108.131:34491)
0806 01:05:27.206350 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:27.206354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:27.206456 (+ 102us) server_negotiation.cc:407] Connection header received
0806 01:05:27.206630 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:27.206636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:27.206744 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:27.206848 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:27.208730 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:27.209521 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:27.211328 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:27.211806 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:27.211896 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:27.216920 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:27.216930 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:27.216944 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:27.216979 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:27.220098 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:27.220778 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:27.220792 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:27.220795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:27.220946 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:27.221385 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:27.221391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:27.221394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:27.221761 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:27.222078 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:27.222430 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 01:05:27.222649 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:05:27.223851 31771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:27.270292 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:27.269843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38125 (local address 127.0.0.1:49370)
0806 01:05:27.270042 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:27.270143 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 01:05:27.271361 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:27.270994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38125 (local address 127.0.0.1:49386)
0806 01:05:27.271196 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:27.271261 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 01:05:27.342171 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 31596
I20260806 01:05:27.365971 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 31047
I20260806 01:05:27.367178 31142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:27.474982 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 31047
2026-08-06T01:05:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4514 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T01:05:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:27Z Disabled control of system clock
I20260806 01:05:27.583983 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:39153
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:39153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:05:27.908578 31862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:27.908859 31862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:27.908939 31862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:27.916988 31862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:27.917114 31862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:27.917155 31862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:27.917214 31862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:27.917259 31862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:27.926090 31862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:39153
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:39153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:27.927294 31862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:27.929205 31862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:27.941017 31870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:27.941067 31871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:27.940946 31874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:27.942456 31862 server_base.cc:1034] running on GCE node
I20260806 01:05:27.943226 31862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:27.944640 31862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:27.945866 31862 hybrid_clock.cc:648] HybridClock initialized: now 1785978327945830 us; error 59 us; skew 500 ppm
I20260806 01:05:27.946201 31862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:27.951890 31862 webserver.cc:460] Webserver started at http://127.5.108.190:36325/ using document root <none> and password file <none>
I20260806 01:05:27.952579 31862 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:27.952718 31862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:27.953033 31862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:27.956054 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "b5dd815671db49f5b1dbe0ac076d3645"
format_stamp: "Formatted at 2026-08-06 01:05:27 on dist-test-slave-x4qh"
server_key: "afbaabaa440b81f1d63851ea0a459749"
server_key_iv: "6e6443fc3f01080ac42829d475453b29"
server_key_version: "encryptionkey@0"
I20260806 01:05:27.956861 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "b5dd815671db49f5b1dbe0ac076d3645"
format_stamp: "Formatted at 2026-08-06 01:05:27 on dist-test-slave-x4qh"
server_key: "afbaabaa440b81f1d63851ea0a459749"
server_key_iv: "6e6443fc3f01080ac42829d475453b29"
server_key_version: "encryptionkey@0"
I20260806 01:05:27.961987 31862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:05:27.965245 31881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:27.966497 31862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:05:27.966681 31862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "b5dd815671db49f5b1dbe0ac076d3645"
format_stamp: "Formatted at 2026-08-06 01:05:27 on dist-test-slave-x4qh"
server_key: "afbaabaa440b81f1d63851ea0a459749"
server_key_iv: "6e6443fc3f01080ac42829d475453b29"
server_key_version: "encryptionkey@0"
I20260806 01:05:27.966889 31862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:27.979328 31862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:27.980172 31862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:27.980481 31862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:27.999501 31862 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:39153
I20260806 01:05:27.999586 31949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:39153 every 8 connection(s)
I20260806 01:05:28.000856 31862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:28.004582 31950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:28.006371 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 31862
I20260806 01:05:28.006568 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:28.007225 5554 external_mini_cluster.cc:1428] Setting key 859081806e21abdbfc127bc0206fbd63
I20260806 01:05:28.017462 31950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645: Bootstrap starting.
I20260806 01:05:28.020155 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:28.009283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.5.108.190:39153)
0806 01:05:28.010197 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:28.010210 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:28.010246 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:05:28.011843 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:28.011863 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:28.012278 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:28.012642 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:28.013154 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:28.014379 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:28.015783 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:28.016409 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:28.016597 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:28.017878 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:28.017886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:28.017897 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:28.018409 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:28.018495 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:28.018514 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:28.018651 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:28.018904 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 01:05:28.019189 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":342,"threads_started":1}
I20260806 01:05:28.021100 31950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:28.022204 31950 log.cc:824] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:28.024899 31950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645: No bootstrap required, opened a new log
I20260806 01:05:28.031952 31950 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5dd815671db49f5b1dbe0ac076d3645" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39153 } }
I20260806 01:05:28.032306 31950 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:28.032429 31950 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5dd815671db49f5b1dbe0ac076d3645, State: Initialized, Role: FOLLOWER
I20260806 01:05:28.033013 31950 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5dd815671db49f5b1dbe0ac076d3645" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39153 } }
I20260806 01:05:28.033260 31950 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:28.033391 31950 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:28.033571 31950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:28.035712 31950 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5dd815671db49f5b1dbe0ac076d3645" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39153 } }
I20260806 01:05:28.036190 31950 leader_election.cc:302] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5dd815671db49f5b1dbe0ac076d3645; no voters:
I20260806 01:05:28.036752 31950 leader_election.cc:288] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:28.036896 31958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:28.038609 31958 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [term 1 LEADER]: Becoming Leader. State: Replica: b5dd815671db49f5b1dbe0ac076d3645, State: Running, Role: LEADER
I20260806 01:05:28.039008 31950 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:28.039110 31958 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5dd815671db49f5b1dbe0ac076d3645" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39153 } }
I20260806 01:05:28.042054 31961 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5dd815671db49f5b1dbe0ac076d3645. Latest consensus state: current_term: 1 leader_uuid: "b5dd815671db49f5b1dbe0ac076d3645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5dd815671db49f5b1dbe0ac076d3645" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39153 } } }
I20260806 01:05:28.042158 31960 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5dd815671db49f5b1dbe0ac076d3645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5dd815671db49f5b1dbe0ac076d3645" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39153 } } }
I20260806 01:05:28.042454 31961 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:28.042547 31960 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:28.048012 31969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:28.053257 31969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:28.061805 31969 catalog_manager.cc:1347] Generated new cluster ID: d973d1a4075242b0a8b0378dbf65c10e
I20260806 01:05:28.061899 31969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:28.077208 31969 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:28.078462 31969 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:28.090122 31969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645: Generated new TSK 0
I20260806 01:05:28.091159 31969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:05:28.106863 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:39153
--builtin_ntp_servers=127.5.108.148:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:05:28.453557 31988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:28.453850 31988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:28.454039 31988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:28.460968 31988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:28.461095 31988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:28.461252 31988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:05:28.470062 31988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:39153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:28.471314 31988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:28.472959 31988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:28.485239 31996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:28.485245 31999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:28.485685 31995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:28.486130 31988 server_base.cc:1034] running on GCE node
I20260806 01:05:28.486661 31988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:28.487599 31988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:28.488808 31988 hybrid_clock.cc:648] HybridClock initialized: now 1785978328488773 us; error 31 us; skew 500 ppm
I20260806 01:05:28.489092 31988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:28.491042 31988 webserver.cc:460] Webserver started at http://127.5.108.129:45871/ using document root <none> and password file <none>
I20260806 01:05:28.491703 31988 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:28.491806 31988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:28.492112 31988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:28.494606 31988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "06b67b6364a749879726c56795861b92"
format_stamp: "Formatted at 2026-08-06 01:05:28 on dist-test-slave-x4qh"
server_key: "559214bd8ffab2dd75373545906c740b"
server_key_iv: "6c9eaed08b63cd45e64935bb43148514"
server_key_version: "encryptionkey@0"
I20260806 01:05:28.495347 31988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "06b67b6364a749879726c56795861b92"
format_stamp: "Formatted at 2026-08-06 01:05:28 on dist-test-slave-x4qh"
server_key: "559214bd8ffab2dd75373545906c740b"
server_key_iv: "6c9eaed08b63cd45e64935bb43148514"
server_key_version: "encryptionkey@0"
I20260806 01:05:28.500028 31988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:05:28.503247 32010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:28.504405 31988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:05:28.504571 31988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "06b67b6364a749879726c56795861b92"
format_stamp: "Formatted at 2026-08-06 01:05:28 on dist-test-slave-x4qh"
server_key: "559214bd8ffab2dd75373545906c740b"
server_key_iv: "6c9eaed08b63cd45e64935bb43148514"
server_key_version: "encryptionkey@0"
I20260806 01:05:28.504771 31988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:28.525046 31988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:28.525743 31988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:28.526036 31988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:28.526866 31988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:28.528292 31988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:28.528429 31988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:28.528545 31988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:28.528640 31988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:28.560194 31988 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:37153
I20260806 01:05:28.560272 32171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:37153 every 8 connection(s)
I20260806 01:05:28.561362 31988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:05:28.562192 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 31988
I20260806 01:05:28.562381 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:05:28.562985 5554 external_mini_cluster.cc:1428] Setting key 7fb83e97a5d098f75f1d1f6fba465e21
I20260806 01:05:28.566390 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:39153
--builtin_ntp_servers=127.5.108.148:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:05:28.576435 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:28.564279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:46513 (local address 127.5.108.190:39153)
0806 01:05:28.564757 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:28.564763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:28.566174 (+ 1411us) server_negotiation.cc:407] Connection header received
0806 01:05:28.569013 (+ 2839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:28.569016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:28.569080 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:28.569175 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:28.570473 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:28.571311 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:28.573649 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:28.574068 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:28.574160 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:28.575753 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:28.575757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:28.575760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:28.575776 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:28.575814 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:28.575825 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:28.575893 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:28.576134 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 01:05:28.576201 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":261,"threads_started":1}
I20260806 01:05:28.577764 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:28.564890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:39153 (local address 127.5.108.129:46513)
0806 01:05:28.566008 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:28.566054 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:28.568725 (+ 2671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:28.569412 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:28.569425 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:28.569531 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:28.570309 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:28.570335 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:28.571479 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:28.571486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:28.573504 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:28.573521 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:28.574287 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:28.574296 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:28.574631 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:28.575413 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:05:28.575419 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:28.575471 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:05:28.575568 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:28.575921 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:28.575934 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:28.575944 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:05:28.576111 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 01:05:28.576475 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":365,"threads_started":1}
I20260806 01:05:28.579327 32172 heartbeater.cc:347] Connected to a master server at 127.5.108.190:39153
I20260806 01:05:28.579677 32172 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:28.580466 32172 heartbeater.cc:511] Master 127.5.108.190:39153 requested a full tablet report, sending...
I20260806 01:05:28.582494 31904 ts_manager.cc:194] Registered new tserver with Master: 06b67b6364a749879726c56795861b92 (127.5.108.129:37153)
I20260806 01:05:28.583897 31904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.129:46513
W20260806 01:05:28.894286 32177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:28.894556 32177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:28.894690 32177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:28.901360 32177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:28.901469 32177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:28.901607 32177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:05:28.910257 32177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:39153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:28.911477 32177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:28.913076 32177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:28.926270 32182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:28.926359 32187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:28.926379 32184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:28.927520 32177 server_base.cc:1034] running on GCE node
I20260806 01:05:28.928097 32177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:28.928941 32177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:28.930140 32177 hybrid_clock.cc:648] HybridClock initialized: now 1785978328930096 us; error 32 us; skew 500 ppm
I20260806 01:05:28.930485 32177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:28.932675 32177 webserver.cc:460] Webserver started at http://127.5.108.130:44425/ using document root <none> and password file <none>
I20260806 01:05:28.933265 32177 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:28.933350 32177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:28.933614 32177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:28.936265 32177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "492a1f2427b44bb2a456b0a205effaa2"
format_stamp: "Formatted at 2026-08-06 01:05:28 on dist-test-slave-x4qh"
server_key: "43ad19bbf6e596a42544c65d1b472ac7"
server_key_iv: "45124632ad37490f4a9599a43db7c570"
server_key_version: "encryptionkey@0"
I20260806 01:05:28.937067 32177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "492a1f2427b44bb2a456b0a205effaa2"
format_stamp: "Formatted at 2026-08-06 01:05:28 on dist-test-slave-x4qh"
server_key: "43ad19bbf6e596a42544c65d1b472ac7"
server_key_iv: "45124632ad37490f4a9599a43db7c570"
server_key_version: "encryptionkey@0"
I20260806 01:05:28.941697 32177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 01:05:28.944762 32196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:28.945943 32177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 01:05:28.946071 32177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "492a1f2427b44bb2a456b0a205effaa2"
format_stamp: "Formatted at 2026-08-06 01:05:28 on dist-test-slave-x4qh"
server_key: "43ad19bbf6e596a42544c65d1b472ac7"
server_key_iv: "45124632ad37490f4a9599a43db7c570"
server_key_version: "encryptionkey@0"
I20260806 01:05:28.946246 32177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:28.977123 32177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:28.977849 32177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:28.978093 32177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:28.979001 32177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:28.980321 32177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:28.980409 32177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:28.980484 32177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:28.980537 32177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:29.014081 32177 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:43301
I20260806 01:05:29.014142 32371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:43301 every 8 connection(s)
I20260806 01:05:29.015219 32177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:05:29.019356 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 32177
I20260806 01:05:29.019558 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:05:29.020102 5554 external_mini_cluster.cc:1428] Setting key 69873391dccfbc8e0f6eec77316d00ed
I20260806 01:05:29.023108 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:39153
--builtin_ntp_servers=127.5.108.148:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:05:29.028787 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.017729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:43673 (local address 127.5.108.190:39153)
0806 01:05:29.017958 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.017965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.019328 (+ 1363us) server_negotiation.cc:407] Connection header received
0806 01:05:29.021478 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.021482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.021541 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.021621 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:29.022849 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.023704 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.026092 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.026451 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.026542 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.028099 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:29.028104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:29.028107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:29.028124 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:29.028156 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:29.028167 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:29.028236 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.028503 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.028572 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:05:29.030041 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.018253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:39153 (local address 127.5.108.130:43673)
0806 01:05:29.019163 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:29.019207 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:29.021190 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:29.021882 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:29.021890 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:29.021967 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:29.022677 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.022700 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.023846 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.023854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.025946 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.025970 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.026663 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.026672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.026988 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.027754 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:05:29.027759 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:29.027817 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:05:29.027909 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:29.028267 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:29.028280 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:29.028290 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:05:29.028489 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 01:05:29.028855 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":260,"threads_started":1}
I20260806 01:05:29.031436 32372 heartbeater.cc:347] Connected to a master server at 127.5.108.190:39153
I20260806 01:05:29.031718 32372 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:29.032341 32372 heartbeater.cc:511] Master 127.5.108.190:39153 requested a full tablet report, sending...
I20260806 01:05:29.033687 31904 ts_manager.cc:194] Registered new tserver with Master: 492a1f2427b44bb2a456b0a205effaa2 (127.5.108.130:43301)
I20260806 01:05:29.034754 31904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.130:43673
W20260806 01:05:29.349957 32376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:29.350221 32376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:29.350391 32376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:29.356861 32376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:29.356990 32376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:29.357165 32376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:05:29.365828 32376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:39153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:29.367007 32376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:29.368835 32376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:29.381347 32384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:29.381559 32383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:29.381507 32386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:29.382405 32376 server_base.cc:1034] running on GCE node
I20260806 01:05:29.383036 32376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:29.383982 32376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:29.385186 32376 hybrid_clock.cc:648] HybridClock initialized: now 1785978329385141 us; error 32 us; skew 500 ppm
I20260806 01:05:29.385481 32376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:29.387607 32376 webserver.cc:460] Webserver started at http://127.5.108.131:43211/ using document root <none> and password file <none>
I20260806 01:05:29.388257 32376 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:29.388372 32376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:29.388715 32376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:29.391227 32376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "9359d75a6efe43109f88fdf069b7c0ee"
format_stamp: "Formatted at 2026-08-06 01:05:29 on dist-test-slave-x4qh"
server_key: "93ea6a68ea3abb8c65605acc09542ee2"
server_key_iv: "6dc53010316ff36c0f84d33a4b4ab4c6"
server_key_version: "encryptionkey@0"
I20260806 01:05:29.392019 32376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "9359d75a6efe43109f88fdf069b7c0ee"
format_stamp: "Formatted at 2026-08-06 01:05:29 on dist-test-slave-x4qh"
server_key: "93ea6a68ea3abb8c65605acc09542ee2"
server_key_iv: "6dc53010316ff36c0f84d33a4b4ab4c6"
server_key_version: "encryptionkey@0"
I20260806 01:05:29.396575 32376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 01:05:29.399789 32393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:29.400964 32376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260806 01:05:29.401134 32376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "9359d75a6efe43109f88fdf069b7c0ee"
format_stamp: "Formatted at 2026-08-06 01:05:29 on dist-test-slave-x4qh"
server_key: "93ea6a68ea3abb8c65605acc09542ee2"
server_key_iv: "6dc53010316ff36c0f84d33a4b4ab4c6"
server_key_version: "encryptionkey@0"
I20260806 01:05:29.401387 32376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:29.426131 32376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:29.426806 32376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:29.427120 32376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:29.428112 32376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:29.429461 32376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:29.429581 32376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:29.429726 32376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:29.429821 32376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:29.461549 32376 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:33939
I20260806 01:05:29.461624 32560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:33939 every 8 connection(s)
I20260806 01:05:29.462699 32376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:05:29.463788 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 32376
I20260806 01:05:29.463953 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:05:29.464530 5554 external_mini_cluster.cc:1428] Setting key b9c04042c01091a64f4a70e6237e04c8
I20260806 01:05:29.474753 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.465276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:46095 (local address 127.5.108.190:39153)
0806 01:05:29.465462 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.465469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.466820 (+ 1351us) server_negotiation.cc:407] Connection header received
0806 01:05:29.468719 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.468724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.468822 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.468928 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:29.469930 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.470754 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.472285 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.472663 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.472808 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.474004 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:29.474009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:29.474013 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:29.474035 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:29.474064 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:29.474074 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:29.474157 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.474358 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.474441 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 01:05:29.475697 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.465879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:39153 (local address 127.5.108.131:46095)
0806 01:05:29.466681 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:29.466725 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:29.468455 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:29.469164 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:29.469171 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:29.469265 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:29.469756 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.469791 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.470915 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.470919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.472132 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.472144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.472901 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.472909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.473113 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.473732 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:05:29.473734 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:29.473770 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:05:29.473835 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:29.474159 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:29.474171 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:29.474179 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 01:05:29.474346 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 01:05:29.474672 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":287,"threads_started":1}
I20260806 01:05:29.476799 32561 heartbeater.cc:347] Connected to a master server at 127.5.108.190:39153
I20260806 01:05:29.477061 32561 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:29.477766 32561 heartbeater.cc:511] Master 127.5.108.190:39153 requested a full tablet report, sending...
I20260806 01:05:29.479286 31904 ts_manager.cc:194] Registered new tserver with Master: 9359d75a6efe43109f88fdf069b7c0ee (127.5.108.131:33939)
I20260806 01:05:29.480050 31904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.131:46095
I20260806 01:05:29.491638 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:05:29.500348 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.495302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.5.108.190:39153)
0806 01:05:29.495458 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.495461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.495905 (+ 444us) server_negotiation.cc:407] Connection header received
0806 01:05:29.496016 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.496019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.496081 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.496149 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:29.496543 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.497124 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.498414 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.498812 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.498896 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.499855 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:29.499860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:29.499862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:29.499877 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:29.499905 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:29.499915 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:29.499981 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.500094 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.500156 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 01:05:29.503504 31904 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:05:29.513281 31904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:05:29.535301 32586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.523940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37153 (local address 127.0.0.1:39386)
0806 01:05:29.524615 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:29.524655 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:29.524806 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:29.525975 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:29.525983 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:29.526015 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:29.526467 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.526481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.529017 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.529024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.531843 (+ 2819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.531862 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.534168 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.534182 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.534464 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.534533 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 01:05:29.534638 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 01:05:29.534865 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":315,"threads_started":1}
I20260806 01:05:29.536855 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.524133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39386 (local address 127.5.108.129:37153)
0806 01:05:29.525330 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.525339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.525369 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:05:29.525461 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.525468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.525720 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.526702 (+ 982us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:29.526933 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.528842 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.532118 (+ 3276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.533912 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.535830 (+ 1918us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.536127 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.536199 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.536401 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1051,"thread_start_us":920,"threads_started":1}
I20260806 01:05:29.542896 32070 tablet_service.cc:1467] Processing CreateTablet for tablet 0641889ab51b4881b92350a1fc969fb3 (DEFAULT_TABLE table=test-table [id=639caa56c0434e10a7cc1a55c923da02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:29.544075 32070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0641889ab51b4881b92350a1fc969fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:29.545097 32586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.525499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:33939 (local address 127.0.0.1:42000)
0806 01:05:29.535533 (+ 10034us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:29.535549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:29.535644 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:29.538237 (+ 2593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:29.538243 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:29.538281 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:29.538523 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.538535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.540778 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.540786 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.542518 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.542534 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.544383 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.544389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.544595 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.544626 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 01:05:29.544725 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 01:05:29.544854 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9941,"thread_start_us":13868,"threads_started":1}
I20260806 01:05:29.545267 32590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.525996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.5.108.131:33939)
0806 01:05:29.537585 (+ 11589us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.537597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.537634 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:05:29.537754 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.537764 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.538003 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.538208 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:29.538756 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.540555 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.542715 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.544120 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.544326 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.544596 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.544782 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.544942 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11390,"thread_start_us":11844,"threads_started":1}
I20260806 01:05:29.548018 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.526031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:43301 (local address 127.0.0.1:42678)
0806 01:05:29.539301 (+ 13270us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:29.539320 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:29.539555 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:29.540120 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:29.540124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:29.540147 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:29.540526 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.540536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.542387 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.542392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.544460 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.544474 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.546112 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.546118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.547450 (+ 1332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.547508 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 01:05:29.547604 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 01:05:29.547717 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13144,"thread_start_us":12998,"threads_started":1}
I20260806 01:05:29.548800 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.526729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.5.108.130:43301)
0806 01:05:29.538863 (+ 12134us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.538874 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.539447 (+ 573us) server_negotiation.cc:407] Connection header received
0806 01:05:29.539796 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.539801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.539950 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.540127 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:29.540674 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.542231 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.544709 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.545931 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.546059 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.546274 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.548323 (+ 2049us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.548506 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11982,"thread_start_us":11879,"threads_started":1}
I20260806 01:05:29.548422 32474 tablet_service.cc:1467] Processing CreateTablet for tablet 0641889ab51b4881b92350a1fc969fb3 (DEFAULT_TABLE table=test-table [id=639caa56c0434e10a7cc1a55c923da02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:29.549830 32474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0641889ab51b4881b92350a1fc969fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:29.552083 32268 tablet_service.cc:1467] Processing CreateTablet for tablet 0641889ab51b4881b92350a1fc969fb3 (DEFAULT_TABLE table=test-table [id=639caa56c0434e10a7cc1a55c923da02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:29.553303 32268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0641889ab51b4881b92350a1fc969fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:29.557983 32595 tablet_bootstrap.cc:492] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92: Bootstrap starting.
I20260806 01:05:29.562211 32595 tablet_bootstrap.cc:654] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:29.565287 32595 log.cc:824] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:29.567021 32596 tablet_bootstrap.cc:492] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: Bootstrap starting.
I20260806 01:05:29.572896 32596 tablet_bootstrap.cc:654] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:29.574194 32596 log.cc:824] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:29.577225 32595 tablet_bootstrap.cc:492] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92: No bootstrap required, opened a new log
I20260806 01:05:29.577445 32596 tablet_bootstrap.cc:492] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: No bootstrap required, opened a new log
I20260806 01:05:29.577533 32595 ts_tablet_manager.cc:1397] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.015s
I20260806 01:05:29.577801 32596 ts_tablet_manager.cc:1397] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20260806 01:05:29.578035 32598 tablet_bootstrap.cc:492] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: Bootstrap starting.
I20260806 01:05:29.580705 32598 tablet_bootstrap.cc:654] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:29.581561 32598 log.cc:824] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:29.583865 32598 tablet_bootstrap.cc:492] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: No bootstrap required, opened a new log
I20260806 01:05:29.584105 32598 ts_tablet_manager.cc:1397] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 01:05:29.586474 32172 heartbeater.cc:503] Master 127.5.108.190:39153 was elected leader, sending a full tablet report...
I20260806 01:05:29.587769 32595 raft_consensus.cc:348] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } }
I20260806 01:05:29.588276 32595 raft_consensus.cc:374] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:29.588433 32595 raft_consensus.cc:729] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b67b6364a749879726c56795861b92, State: Initialized, Role: FOLLOWER
I20260806 01:05:29.588884 32596 raft_consensus.cc:348] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } }
I20260806 01:05:29.589109 32595 consensus_queue.cc:260] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } }
I20260806 01:05:29.589355 32596 raft_consensus.cc:374] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:29.589541 32596 raft_consensus.cc:729] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9359d75a6efe43109f88fdf069b7c0ee, State: Initialized, Role: FOLLOWER
I20260806 01:05:29.590371 32596 consensus_queue.cc:260] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } }
I20260806 01:05:29.592073 32595 ts_tablet_manager.cc:1428] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260806 01:05:29.592965 32598 raft_consensus.cc:348] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } }
I20260806 01:05:29.593199 32561 heartbeater.cc:503] Master 127.5.108.190:39153 was elected leader, sending a full tablet report...
I20260806 01:05:29.593381 32598 raft_consensus.cc:374] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:29.593519 32598 raft_consensus.cc:729] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492a1f2427b44bb2a456b0a205effaa2, State: Initialized, Role: FOLLOWER
I20260806 01:05:29.593922 32596 ts_tablet_manager.cc:1428] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: Time spent starting tablet: real 0.016s user 0.000s sys 0.014s
I20260806 01:05:29.594298 32598 consensus_queue.cc:260] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } }
I20260806 01:05:29.595441 32372 heartbeater.cc:503] Master 127.5.108.190:39153 was elected leader, sending a full tablet report...
I20260806 01:05:29.595901 32598 ts_tablet_manager.cc:1428] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
W20260806 01:05:29.714385 32563 tablet.cc:2369] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:05:29.745772 32605 raft_consensus.cc:482] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:29.746124 32605 raft_consensus.cc:504] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } }
I20260806 01:05:29.747931 32605 leader_election.cc:288] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06b67b6364a749879726c56795861b92 (127.5.108.129:37153), 9359d75a6efe43109f88fdf069b7c0ee (127.5.108.131:33939)
I20260806 01:05:29.756296 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.748594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:45109 (local address 127.5.108.129:37153)
0806 01:05:29.748796 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.748801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.749223 (+ 422us) server_negotiation.cc:407] Connection header received
0806 01:05:29.749465 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.749472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.749571 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.749690 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:29.750487 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.751631 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.754141 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.755328 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.755524 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.755606 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.755894 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.756017 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:05:29.756292 32590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.749623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:53877 (local address 127.5.108.131:33939)
0806 01:05:29.749838 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.749844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.750015 (+ 171us) server_negotiation.cc:407] Connection header received
0806 01:05:29.750183 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.750186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.750245 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.750325 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:29.750921 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.751963 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.754240 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.755474 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.755633 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.755690 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.755936 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.756051 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:05:29.756304 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.749248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:33939 (local address 127.5.108.130:53877)
0806 01:05:29.749886 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:29.749902 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:29.750061 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:29.750356 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:29.750362 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:29.750390 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:29.750730 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.750751 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.752124 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.752129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.754066 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.754079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.755637 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.755642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.755787 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.755825 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:05:29.755895 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 01:05:29.755986 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":363,"threads_started":1}
I20260806 01:05:29.756343 32606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.748390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37153 (local address 127.5.108.130:45109)
0806 01:05:29.749115 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:29.749134 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:29.749277 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:29.749829 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:29.749834 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:29.749860 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:29.750324 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.750337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.751751 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.751755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.753936 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:29.753955 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.755516 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.755520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:29.755711 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.755748 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:05:29.755837 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 01:05:29.755980 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"spinlock_wait_cycles":20352,"thread_start_us":413,"threads_started":1}
I20260806 01:05:29.757484 32498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0641889ab51b4881b92350a1fc969fb3" candidate_uuid: "492a1f2427b44bb2a456b0a205effaa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9359d75a6efe43109f88fdf069b7c0ee" is_pre_election: true
I20260806 01:05:29.757663 32103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0641889ab51b4881b92350a1fc969fb3" candidate_uuid: "492a1f2427b44bb2a456b0a205effaa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b67b6364a749879726c56795861b92" is_pre_election: true
I20260806 01:05:29.758116 32498 raft_consensus.cc:2454] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492a1f2427b44bb2a456b0a205effaa2 in term 0.
I20260806 01:05:29.758226 32103 raft_consensus.cc:2454] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492a1f2427b44bb2a456b0a205effaa2 in term 0.
I20260806 01:05:29.758816 32200 leader_election.cc:302] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492a1f2427b44bb2a456b0a205effaa2, 9359d75a6efe43109f88fdf069b7c0ee; no voters:
I20260806 01:05:29.759176 32605 raft_consensus.cc:2781] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:05:29.759338 32605 raft_consensus.cc:482] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:05:29.759495 32605 raft_consensus.cc:3037] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:29.761598 32605 raft_consensus.cc:504] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } }
I20260806 01:05:29.762167 32605 leader_election.cc:288] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [CANDIDATE]: Term 1 election: Requested vote from peers 06b67b6364a749879726c56795861b92 (127.5.108.129:37153), 9359d75a6efe43109f88fdf069b7c0ee (127.5.108.131:33939)
I20260806 01:05:29.762620 32103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0641889ab51b4881b92350a1fc969fb3" candidate_uuid: "492a1f2427b44bb2a456b0a205effaa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b67b6364a749879726c56795861b92"
I20260806 01:05:29.762728 32498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0641889ab51b4881b92350a1fc969fb3" candidate_uuid: "492a1f2427b44bb2a456b0a205effaa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9359d75a6efe43109f88fdf069b7c0ee"
I20260806 01:05:29.762809 32103 raft_consensus.cc:3037] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:29.762928 32498 raft_consensus.cc:3037] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:29.765019 32103 raft_consensus.cc:2454] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492a1f2427b44bb2a456b0a205effaa2 in term 1.
I20260806 01:05:29.765093 32498 raft_consensus.cc:2454] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492a1f2427b44bb2a456b0a205effaa2 in term 1.
I20260806 01:05:29.765408 32200 leader_election.cc:302] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b67b6364a749879726c56795861b92, 492a1f2427b44bb2a456b0a205effaa2; no voters:
I20260806 01:05:29.765697 32605 raft_consensus.cc:2781] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:29.766202 32605 raft_consensus.cc:686] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 1 LEADER]: Becoming Leader. State: Replica: 492a1f2427b44bb2a456b0a205effaa2, State: Running, Role: LEADER
I20260806 01:05:29.766606 32605 consensus_queue.cc:237] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } }
W20260806 01:05:29.767336 32373 tablet.cc:2369] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:05:29.771481 31903 catalog_manager.cc:5495] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 492a1f2427b44bb2a456b0a205effaa2 (127.5.108.130). New cstate: current_term: 1 leader_uuid: "492a1f2427b44bb2a456b0a205effaa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "492a1f2427b44bb2a456b0a205effaa2" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 43301 } health_report { overall_health: HEALTHY } } }
W20260806 01:05:29.814313 32173 tablet.cc:2369] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:05:29.827956 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.822505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.5.108.130:43301)
0806 01:05:29.822745 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.822752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.822852 (+ 100us) server_negotiation.cc:407] Connection header received
0806 01:05:29.823094 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.823098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.823213 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.823367 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:05:29.823766 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.824483 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.825848 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.826256 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.826389 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.826848 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:05:29.827337 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:05:29.827532 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.827613 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.827691 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:05:29.836211 32498 raft_consensus.cc:1261] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [term 1 FOLLOWER]: Refusing update from remote peer 492a1f2427b44bb2a456b0a205effaa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:29.836382 32103 raft_consensus.cc:1261] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [term 1 FOLLOWER]: Refusing update from remote peer 492a1f2427b44bb2a456b0a205effaa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:29.837265 32610 consensus_queue.cc:1035] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9359d75a6efe43109f88fdf069b7c0ee" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:29.837644 32605 consensus_queue.cc:1035] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b67b6364a749879726c56795861b92" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 37153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:29.851435 32613 mvcc.cc:205] Tried to move back new op lower bound from 7315367238997946368 to 7315367238728298496. Current Snapshot: MvccSnapshot[applied={T|T < 7315367238997946368}]
I20260806 01:05:29.864707 31903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:29.864985 31903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:29.868672 31903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5dd815671db49f5b1dbe0ac076d3645: Sending DeleteTablet for 3 replicas of tablet 0641889ab51b4881b92350a1fc969fb3
I20260806 01:05:29.869843 32474 tablet_service.cc:1514] Processing DeleteTablet for tablet 0641889ab51b4881b92350a1fc969fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:29 UTC) from {username='slave'} at 127.0.0.1:42000
I20260806 01:05:29.869845 32070 tablet_service.cc:1514] Processing DeleteTablet for tablet 0641889ab51b4881b92350a1fc969fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:29 UTC) from {username='slave'} at 127.0.0.1:39386
I20260806 01:05:29.869908 32268 tablet_service.cc:1514] Processing DeleteTablet for tablet 0641889ab51b4881b92350a1fc969fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:29 UTC) from {username='slave'} at 127.0.0.1:42678
I20260806 01:05:29.874372 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 31988
I20260806 01:05:29.883334 32633 tablet_replica.cc:330] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: stopping tablet replica
I20260806 01:05:29.883762 32633 raft_consensus.cc:2229] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:05:29.884361 32633 raft_consensus.cc:2258] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:29.886312 32633 ts_tablet_manager.cc:1905] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:29.893770 32633 ts_tablet_manager.cc:1918] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:29.893992 32633 log.cc:1196] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/0641889ab51b4881b92350a1fc969fb3
I20260806 01:05:29.894588 32633 ts_tablet_manager.cc:1939] T 0641889ab51b4881b92350a1fc969fb3 P 492a1f2427b44bb2a456b0a205effaa2: Deleting consensus metadata
I20260806 01:05:29.896375 31886 catalog_manager.cc:4841] TS 492a1f2427b44bb2a456b0a205effaa2 (127.5.108.130:43301): tablet 0641889ab51b4881b92350a1fc969fb3 (table test-table [id=639caa56c0434e10a7cc1a55c923da02]) successfully deleted
I20260806 01:05:29.900574 32631 tablet_replica.cc:330] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: stopping tablet replica
I20260806 01:05:29.901212 32630 tablet_replica.cc:330] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92: stopping tablet replica
I20260806 01:05:29.901784 32631 raft_consensus.cc:2229] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:29.902277 32630 raft_consensus.cc:2229] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:29.902791 32630 raft_consensus.cc:2258] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:29.903496 32631 raft_consensus.cc:2258] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:29.904460 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:29.876706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39396 (local address 127.5.108.129:37153)
0806 01:05:29.876925 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:29.876933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:29.894571 (+ 17638us) server_negotiation.cc:407] Connection header received
0806 01:05:29.895128 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:29.895137 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:29.895300 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:29.895406 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:29.896216 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.897738 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.900678 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:29.901212 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:29.901354 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:29.902644 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:29.902652 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:29.902664 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:29.903397 (+ 733us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:29.903456 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:29.903480 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:29.903589 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:29.903959 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 01:05:29.904046 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:05:29.905622 32630 ts_tablet_manager.cc:1905] T 0641889ab51b4881b92350a1fc969fb3 P 06b67b6364a749879726c56795861b92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:29.905725 32631 ts_tablet_manager.cc:1905] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:29.906086 32167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:29.913089 32631 ts_tablet_manager.cc:1918] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:29.913254 32631 log.cc:1196] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/0641889ab51b4881b92350a1fc969fb3
I20260806 01:05:29.913713 32631 ts_tablet_manager.cc:1939] T 0641889ab51b4881b92350a1fc969fb3 P 9359d75a6efe43109f88fdf069b7c0ee: Deleting consensus metadata
I20260806 01:05:29.915201 31885 catalog_manager.cc:4841] TS 9359d75a6efe43109f88fdf069b7c0ee (127.5.108.131:33939): tablet 0641889ab51b4881b92350a1fc969fb3 (table test-table [id=639caa56c0434e10a7cc1a55c923da02]) successfully deleted
I20260806 01:05:30.035831 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 31988
W20260806 01:05:30.062801 31885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.108.129:37153 (error 108)
I20260806 01:05:30.063582 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 32177
I20260806 01:05:30.065265 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:30.064000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37153 (local address 127.0.0.1:39412)
0806 01:05:30.064911 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:30.065043 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":686,"threads_started":1}
W20260806 01:05:30.065560 31885 catalog_manager.cc:4584] TS 06b67b6364a749879726c56795861b92 (127.5.108.129:37153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0641889ab51b4881b92350a1fc969fb3: Network error: Client connection negotiation failed: client connection to 127.5.108.129:37153: connect: Connection refused (error 111)
I20260806 01:05:30.071350 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:30.064351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42706 (local address 127.5.108.130:43301)
0806 01:05:30.064552 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:30.064559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:30.064969 (+ 410us) server_negotiation.cc:407] Connection header received
0806 01:05:30.065030 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:30.065034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:30.065123 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:30.065242 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:30.065726 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:30.066530 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:30.068174 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:30.068639 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:30.068791 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:30.069879 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:30.069888 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:30.069922 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:30.070538 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:30.070583 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:30.070603 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:30.070735 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:30.070887 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 01:05:30.070987 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:05:30.072130 32366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:30.119112 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:30.118756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37153 (local address 127.0.0.1:39420)
0806 01:05:30.118936 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:30.119001 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 01:05:30.120057 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:30.119737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37153 (local address 127.0.0.1:39424)
0806 01:05:30.119871 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:30.119936 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 01:05:30.180932 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:30.180236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37153 (local address 127.0.0.1:39436)
0806 01:05:30.180527 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:30.180638 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260806 01:05:30.181932 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 32177
I20260806 01:05:30.182013 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:30.181575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37153 (local address 127.0.0.1:39442)
0806 01:05:30.181779 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:30.181868 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 01:05:30.210126 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 32376
I20260806 01:05:30.216965 32590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:30.210918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.5.108.131:33939)
0806 01:05:30.211096 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:30.211104 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:30.211128 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:05:30.211335 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:30.211343 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:30.211482 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:30.211587 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:30.211997 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:30.212664 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:30.213890 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:30.214320 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:30.214464 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:30.215714 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:30.215722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:30.215737 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:30.216270 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:30.216314 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:30.216338 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:30.216458 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:30.216571 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 01:05:30.216668 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:05:30.217684 32553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:30.271921 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:30.271341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37153 (local address 127.0.0.1:39452)
0806 01:05:30.271628 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:30.271728 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 01:05:30.273020 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:30.272583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:37153 (local address 127.0.0.1:58340)
0806 01:05:30.272788 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:30.272867 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.129:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 01:05:30.301538 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 32376
I20260806 01:05:30.328119 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 31862
I20260806 01:05:30.329362 31944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:30.431221 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 31862
2026-08-06T01:05:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2940 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:30 dist-test-slave-x4qh krb5kdc[32649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:30 dist-test-slave-x4qh krb5kdc[32649](info): set up 2 sockets
Aug 06 01:05:30 dist-test-slave-x4qh krb5kdc[32649](info): commencing operation
krb5kdc: starting...
W20260806 01:05:31.708376 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:31 dist-test-slave-x4qh krb5kdc[32649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:05:31.986653 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44435
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:44435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:32.310369 32670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:32.310704 32670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:32.310876 32670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:32.317754 32670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:32.317902 32670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:32.318020 32670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:32.318120 32670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:32.318210 32670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:32.327517 32670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:44435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:32.328806 32670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:32.330107 32670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:32.341928 32676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:32.341992 32675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:32.342022 32680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:32.342978 32670 server_base.cc:1034] running on GCE node
I20260806 01:05:32.343796 32670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:32.345098 32670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:32.346339 32670 hybrid_clock.cc:648] HybridClock initialized: now 1785978332346282 us; error 43 us; skew 500 ppm
I20260806 01:05:32.346692 32670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:32 dist-test-slave-x4qh krb5kdc[32649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978332, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:32.349797 32670 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:05:32.351255 32670 webserver.cc:460] Webserver started at http://127.5.108.190:46553/ using document root <none> and password file <none>
I20260806 01:05:32.351935 32670 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:32.352087 32670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:32.352437 32670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:32.355199 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "5426c60fad0442609e869bd91c22f01d"
format_stamp: "Formatted at 2026-08-06 01:05:32 on dist-test-slave-x4qh"
server_key: "5577e8e926aad6db27d7ff9f188fddae"
server_key_iv: "03f8eeffb2e9a77a6e10085c7d2790ba"
server_key_version: "encryptionkey@0"
I20260806 01:05:32.356047 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "5426c60fad0442609e869bd91c22f01d"
format_stamp: "Formatted at 2026-08-06 01:05:32 on dist-test-slave-x4qh"
server_key: "5577e8e926aad6db27d7ff9f188fddae"
server_key_iv: "03f8eeffb2e9a77a6e10085c7d2790ba"
server_key_version: "encryptionkey@0"
I20260806 01:05:32.361052 32670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:05:32.364483 32693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:32.365726 32670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:05:32.365916 32670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "5426c60fad0442609e869bd91c22f01d"
format_stamp: "Formatted at 2026-08-06 01:05:32 on dist-test-slave-x4qh"
server_key: "5577e8e926aad6db27d7ff9f188fddae"
server_key_iv: "03f8eeffb2e9a77a6e10085c7d2790ba"
server_key_version: "encryptionkey@0"
I20260806 01:05:32.366113 32670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:32.392195 32670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:32.396034 32670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:32.396337 32670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:32.414301 32670 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:44435
I20260806 01:05:32.414331 308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:44435 every 8 connection(s)
I20260806 01:05:32.415575 32670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:32.419057 311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:32.421341 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 32670
I20260806 01:05:32.421561 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:32.422029 5554 external_mini_cluster.cc:1428] Setting key 7f5dc2c30c80fcf10dfdd5b532a5f784
I20260806 01:05:32.431521 311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d: Bootstrap starting.
I20260806 01:05:32.434718 311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:32.435678 311 log.cc:824] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:05:32 dist-test-slave-x4qh krb5kdc[32649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:32.438797 311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d: No bootstrap required, opened a new log
I20260806 01:05:32.445410 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:32.423857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57316 (local address 127.5.108.190:44435)
0806 01:05:32.424700 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:32.424714 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:32.424772 (+ 58us) server_negotiation.cc:407] Connection header received
0806 01:05:32.426954 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:32.426979 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:32.427416 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:32.427881 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:32.429195 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:32.431123 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:32.432532 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:32.433586 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:32.433763 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:32.438487 (+ 4724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:32.438495 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:32.438508 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:32.438545 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:32.441599 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:32.442374 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:32.442379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:32.442384 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:32.442526 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:32.442913 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:32.442916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:32.442918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:32.443474 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:32.443683 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:32.444087 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 01:05:32.444461 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":341,"threads_started":1}
I20260806 01:05:32.446877 311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5426c60fad0442609e869bd91c22f01d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44435 } }
I20260806 01:05:32.447216 311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:32.447345 311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5426c60fad0442609e869bd91c22f01d, State: Initialized, Role: FOLLOWER
I20260806 01:05:32.447948 311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5426c60fad0442609e869bd91c22f01d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44435 } }
I20260806 01:05:32.448199 311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:32.448370 311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:32.448557 311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:32.450448 311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5426c60fad0442609e869bd91c22f01d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44435 } }
I20260806 01:05:32.450873 311 leader_election.cc:302] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5426c60fad0442609e869bd91c22f01d; no voters:
I20260806 01:05:32.451469 311 leader_election.cc:288] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:32.451582 318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:32.452234 318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [term 1 LEADER]: Becoming Leader. State: Replica: 5426c60fad0442609e869bd91c22f01d, State: Running, Role: LEADER
I20260806 01:05:32.452785 318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5426c60fad0442609e869bd91c22f01d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44435 } }
I20260806 01:05:32.453433 311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:32.455824 320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5426c60fad0442609e869bd91c22f01d. Latest consensus state: current_term: 1 leader_uuid: "5426c60fad0442609e869bd91c22f01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5426c60fad0442609e869bd91c22f01d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44435 } } }
I20260806 01:05:32.455850 319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5426c60fad0442609e869bd91c22f01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5426c60fad0442609e869bd91c22f01d" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44435 } } }
I20260806 01:05:32.456247 319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:32.456247 320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:32.458570 325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:32.463550 325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:32.472119 325 catalog_manager.cc:1347] Generated new cluster ID: a7d43421ba8645a7bc15364895b08b7e
I20260806 01:05:32.472218 325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:32.483462 325 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:32.485157 325 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:32.495883 325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d: Generated new TSK 0
I20260806 01:05:32.496753 325 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:05:32.574859 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44435
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:32.920827 344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:32.921068 344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:32.921160 344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:32.927809 344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:32.927907 344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:32.928016 344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:05:32.938026 344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:32.939177 344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:32.940591 344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:32.953931 349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:32.954002 350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:32.954028 355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:32.954607 344 server_base.cc:1034] running on GCE node
I20260806 01:05:32.955269 344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:32.956207 344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:32.957406 344 hybrid_clock.cc:648] HybridClock initialized: now 1785978332957374 us; error 28 us; skew 500 ppm
I20260806 01:05:32.957739 344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:32 dist-test-slave-x4qh krb5kdc[32649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978332, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:32.966334 344 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:05:32.967967 344 webserver.cc:460] Webserver started at http://127.5.108.129:46329/ using document root <none> and password file <none>
I20260806 01:05:32.968662 344 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:32.968847 344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:32.969193 344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:32.972231 344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "b9c9a2af7c7a44089a3a2a03260ee74c"
format_stamp: "Formatted at 2026-08-06 01:05:32 on dist-test-slave-x4qh"
server_key: "c38006d5a9f23279601b651be76190fb"
server_key_iv: "158ace31787730eb6c9f11839689ecb3"
server_key_version: "encryptionkey@0"
I20260806 01:05:32.973006 344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "b9c9a2af7c7a44089a3a2a03260ee74c"
format_stamp: "Formatted at 2026-08-06 01:05:32 on dist-test-slave-x4qh"
server_key: "c38006d5a9f23279601b651be76190fb"
server_key_iv: "158ace31787730eb6c9f11839689ecb3"
server_key_version: "encryptionkey@0"
I20260806 01:05:32.978484 344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:05:32.982065 366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:32.983253 344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:05:32.983456 344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "b9c9a2af7c7a44089a3a2a03260ee74c"
format_stamp: "Formatted at 2026-08-06 01:05:32 on dist-test-slave-x4qh"
server_key: "c38006d5a9f23279601b651be76190fb"
server_key_iv: "158ace31787730eb6c9f11839689ecb3"
server_key_version: "encryptionkey@0"
I20260806 01:05:32.983680 344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:33.002928 344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:33.007308 344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:33.007692 344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:33.008765 344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:33.010124 344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:33.010237 344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:33.010404 344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:33.010494 344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:33.044828 344 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:45779
I20260806 01:05:33.044863 534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:45779 every 8 connection(s)
I20260806 01:05:33.046166 344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:05:33.049352 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 344
I20260806 01:05:33.049566 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:05:33.050215 5554 external_mini_cluster.cc:1428] Setting key e9aa2cff83d818534a314f31cd4bbad1
Aug 06 01:05:33 dist-test-slave-x4qh krb5kdc[32649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978332, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:33.066526 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:33.048701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:37159 (local address 127.5.108.190:44435)
0806 01:05:33.049158 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:33.049162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:33.050231 (+ 1069us) server_negotiation.cc:407] Connection header received
0806 01:05:33.052590 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:33.052596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:33.052691 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:33.052812 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:33.054377 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:33.055120 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:33.056604 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:33.057015 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:33.057227 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:33.060963 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:33.060968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:33.060971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:33.061002 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:33.063602 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:33.064268 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:33.064271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:33.064273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:33.064413 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:33.064852 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:33.064856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:33.064857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:33.065099 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:33.065262 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:33.065925 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 01:05:33.066149 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":233,"threads_started":1}
I20260806 01:05:33.067447 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:33.049281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44435 (local address 127.5.108.129:37159)
0806 01:05:33.050084 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:33.050124 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:33.052301 (+ 2177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:33.053081 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:33.053092 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:33.053588 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:33.054211 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:33.054230 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:33.055268 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:33.055273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:33.056458 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:33.056470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:33.057330 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:33.057338 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:33.057576 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:33.058283 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:33.058308 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:33.060755 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:33.063732 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:33.063747 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:33.063760 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:33.064130 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:33.064528 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:33.064533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:33.064536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:33.064718 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:33.065230 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:33.065237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:33.065542 (+ 305us) client_negotiation.cc:769] Sending connection context
0806 01:05:33.065800 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 01:05:33.066313 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":383,"threads_started":1}
I20260806 01:05:33.069042 536 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44435
I20260806 01:05:33.069397 536 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:33.070312 536 heartbeater.cc:511] Master 127.5.108.190:44435 requested a full tablet report, sending...
I20260806 01:05:33.072858 32722 ts_manager.cc:194] Registered new tserver with Master: b9c9a2af7c7a44089a3a2a03260ee74c (127.5.108.129:45779)
I20260806 01:05:33.074990 32722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:37159
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:05:33.118614 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44435
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:33.437899 547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:33.438141 547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:33.438234 547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:33.445020 547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:33.445113 547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:33.445214 547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:05:33.454138 547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:33.455221 547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:33.456359 547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:33.468874 556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:33.468830 558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:33.468910 555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:33.469728 547 server_base.cc:1034] running on GCE node
I20260806 01:05:33.470330 547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:33.471179 547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:33.472407 547 hybrid_clock.cc:648] HybridClock initialized: now 1785978333472364 us; error 54 us; skew 500 ppm
I20260806 01:05:33.472702 547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:33 dist-test-slave-x4qh krb5kdc[32649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978333, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:33.475823 547 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:05:33.477178 547 webserver.cc:460] Webserver started at http://127.5.108.130:43803/ using document root <none> and password file <none>
I20260806 01:05:33.477777 547 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:33.477874 547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:33.478123 547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:33.480775 547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "5e65bdcb2362498b8bb1c4c548290939"
format_stamp: "Formatted at 2026-08-06 01:05:33 on dist-test-slave-x4qh"
server_key: "fc857efd14a728be3e0281da5180993f"
server_key_iv: "87104890a7c56ffc1ee5435e52fdb4eb"
server_key_version: "encryptionkey@0"
I20260806 01:05:33.481585 547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "5e65bdcb2362498b8bb1c4c548290939"
format_stamp: "Formatted at 2026-08-06 01:05:33 on dist-test-slave-x4qh"
server_key: "fc857efd14a728be3e0281da5180993f"
server_key_iv: "87104890a7c56ffc1ee5435e52fdb4eb"
server_key_version: "encryptionkey@0"
I20260806 01:05:33.486092 547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:05:33.489389 573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:33.490481 547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:05:33.490602 547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "5e65bdcb2362498b8bb1c4c548290939"
format_stamp: "Formatted at 2026-08-06 01:05:33 on dist-test-slave-x4qh"
server_key: "fc857efd14a728be3e0281da5180993f"
server_key_iv: "87104890a7c56ffc1ee5435e52fdb4eb"
server_key_version: "encryptionkey@0"
I20260806 01:05:33.490741 547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:33.511735 547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:33.515126 547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:33.515475 547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:33.516338 547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:33.517504 547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:33.517633 547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:33.517764 547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:33.517839 547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:33.550583 547 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:44911
I20260806 01:05:33.550683 751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:44911 every 8 connection(s)
I20260806 01:05:33.552075 547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:05:33.552420 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 547
I20260806 01:05:33.552650 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:05:33.553261 5554 external_mini_cluster.cc:1428] Setting key d6af54d73e8d02941428abf07baab315
Aug 06 01:05:33 dist-test-slave-x4qh krb5kdc[32649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978333, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:33.574106 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:33.554785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:44497 (local address 127.5.108.190:44435)
0806 01:05:33.554968 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:33.554972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:33.556542 (+ 1570us) server_negotiation.cc:407] Connection header received
0806 01:05:33.558708 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:33.558716 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:33.558869 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:33.558993 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:33.560676 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:33.561835 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:33.563245 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:33.563999 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:33.564141 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:33.567752 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:33.567761 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:33.567769 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:33.567813 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:33.570923 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:33.571831 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:33.571834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:33.571836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:33.571968 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:33.572449 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:33.572452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:33.572454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:33.572704 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:33.572816 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:33.573685 (+ 869us) server_negotiation.cc:299] Negotiation successful
0806 01:05:33.573862 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:05:33.575731 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:33.555502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44435 (local address 127.5.108.130:44497)
0806 01:05:33.556362 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:33.556406 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:33.558340 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:33.559278 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:33.559286 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:33.559926 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:33.560522 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:33.560538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:33.561985 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:33.562006 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:33.563099 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:33.563110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:33.564209 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:33.564214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:33.564440 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:33.565120 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:33.565146 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:33.567503 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:33.571057 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:33.571068 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:33.571085 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:33.571721 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:33.572073 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:33.572080 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:33.572087 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:33.572331 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:33.572832 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:33.572842 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:33.573273 (+ 431us) client_negotiation.cc:769] Sending connection context
0806 01:05:33.573612 (+ 339us) client_negotiation.cc:240] Negotiation successful
0806 01:05:33.574221 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":368,"threads_started":1}
I20260806 01:05:33.577231 753 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44435
I20260806 01:05:33.577536 753 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:33.578258 753 heartbeater.cc:511] Master 127.5.108.190:44435 requested a full tablet report, sending...
I20260806 01:05:33.579708 32722 ts_manager.cc:194] Registered new tserver with Master: 5e65bdcb2362498b8bb1c4c548290939 (127.5.108.130:44911)
I20260806 01:05:33.580540 32722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:44497
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:05:33.622053 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44435
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:33.949169 760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:33.949465 760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:33.949635 760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:33.956602 760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:33.956748 760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:33.956913 760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:05:33.966061 760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:33.967322 760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:33.968616 760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:33.982301 767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:33.982406 766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:33.982381 769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:33.983140 760 server_base.cc:1034] running on GCE node
I20260806 01:05:33.983783 760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:33.984683 760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:33.985913 760 hybrid_clock.cc:648] HybridClock initialized: now 1785978333985876 us; error 46 us; skew 500 ppm
I20260806 01:05:33.986220 760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:33 dist-test-slave-x4qh krb5kdc[32649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978333, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:33.989526 760 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:05:33.990983 760 webserver.cc:460] Webserver started at http://127.5.108.131:41419/ using document root <none> and password file <none>
I20260806 01:05:33.991726 760 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:33.991873 760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:33.992193 760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:33.995144 760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c"
format_stamp: "Formatted at 2026-08-06 01:05:33 on dist-test-slave-x4qh"
server_key: "89f2e09c669e30dd3dc788fb187da110"
server_key_iv: "d01fe1f55112df85f369668d0f555eff"
server_key_version: "encryptionkey@0"
I20260806 01:05:33.995980 760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c"
format_stamp: "Formatted at 2026-08-06 01:05:33 on dist-test-slave-x4qh"
server_key: "89f2e09c669e30dd3dc788fb187da110"
server_key_iv: "d01fe1f55112df85f369668d0f555eff"
server_key_version: "encryptionkey@0"
I20260806 01:05:34.001107 760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:05:34.004356 778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:34.005823 760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:05:34.005985 760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c"
format_stamp: "Formatted at 2026-08-06 01:05:33 on dist-test-slave-x4qh"
server_key: "89f2e09c669e30dd3dc788fb187da110"
server_key_iv: "d01fe1f55112df85f369668d0f555eff"
server_key_version: "encryptionkey@0"
I20260806 01:05:34.006178 760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:34.024232 760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:34.028517 760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:34.028827 760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:34.029739 760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:34.030992 760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:34.031119 760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:34.031278 760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:34.031352 760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:34.065757 760 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:38061
I20260806 01:05:34.065843 939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:38061 every 8 connection(s)
I20260806 01:05:34.067036 760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:05:34.069610 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 760
I20260806 01:05:34.069828 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:05:34.070405 5554 external_mini_cluster.cc:1428] Setting key a3d8cab64cb41af717eda2d132578b3a
I20260806 01:05:34.078742 536 heartbeater.cc:503] Master 127.5.108.190:44435 was elected leader, sending a full tablet report...
Aug 06 01:05:34 dist-test-slave-x4qh krb5kdc[32649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978333, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:34.087483 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.069587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:33371 (local address 127.5.108.190:44435)
0806 01:05:34.069821 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.069827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.071301 (+ 1474us) server_negotiation.cc:407] Connection header received
0806 01:05:34.073485 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.073489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.073551 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.073629 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:34.074989 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.075815 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.077566 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.077997 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.078087 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.081689 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:34.081693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:34.081696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:34.081726 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:34.084404 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.085342 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.085349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.085353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.085528 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.085949 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.085952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.085954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.086195 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:34.086308 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.086992 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.087143 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 01:05:34.088481 942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.070098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44435 (local address 127.5.108.131:33371)
0806 01:05:34.070998 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:34.071036 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:34.073202 (+ 2166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:34.073842 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:34.073849 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:34.074317 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:34.074851 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.074868 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.075941 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.075945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.077441 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.077454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.078181 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.078187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.078379 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.079173 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:34.079199 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:34.081514 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:34.084551 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:34.084561 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:34.084575 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:34.085200 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:34.085656 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:34.085661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:34.085664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:34.085852 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:34.086320 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:34.086326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:34.086602 (+ 276us) client_negotiation.cc:769] Sending connection context
0806 01:05:34.086901 (+ 299us) client_negotiation.cc:240] Negotiation successful
0806 01:05:34.087340 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":370,"threads_started":1}
I20260806 01:05:34.089810 940 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44435
I20260806 01:05:34.090085 940 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:34.090749 940 heartbeater.cc:511] Master 127.5.108.190:44435 requested a full tablet report, sending...
I20260806 01:05:34.092120 32722 ts_manager.cc:194] Registered new tserver with Master: ca473b3e89ef4cd3bc3cb8475733fa3c (127.5.108.131:38061)
I20260806 01:05:34.092999 32722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:33371
I20260806 01:05:34.097649 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:05:34.114418 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.101738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.5.108.190:44435)
0806 01:05:34.101897 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.101901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.102372 (+ 471us) server_negotiation.cc:407] Connection header received
0806 01:05:34.102478 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.102481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.102534 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.102609 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:34.104114 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.104820 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.106083 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.106495 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.106582 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.109174 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:34.109178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:34.109191 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:34.109221 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:34.112086 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.112709 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.112712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.112714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.112822 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.113170 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.113173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.113175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.113424 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:34.113600 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.113929 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.114055 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 01:05:34.118067 32722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:05:34.124725 32722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:05:34.148625 957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.136357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:38061 (local address 127.0.0.1:38708)
0806 01:05:34.137259 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:34.137320 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:34.137533 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:34.138876 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:34.138885 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:34.138917 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:34.139444 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.139462 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.141987 (+ 2525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.141995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.144904 (+ 2909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.144919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.147492 (+ 2573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.147502 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.147795 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.147874 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 01:05:34.148014 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 01:05:34.148213 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":427,"threads_started":1}
I20260806 01:05:34.151297 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.136744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.5.108.131:38061)
0806 01:05:34.138194 (+ 1450us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.138202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.138236 (+ 34us) server_negotiation.cc:407] Connection header received
0806 01:05:34.138342 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.138351 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.138593 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.139682 (+ 1089us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:34.140040 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.141793 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.145176 (+ 3383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.147197 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.150464 (+ 3267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.150799 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.150871 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.151040 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1280,"thread_start_us":1557,"threads_started":1}
I20260806 01:05:34.155706 847 tablet_service.cc:1467] Processing CreateTablet for tablet decc3d009ee048f2b36d091edcdf156e (DEFAULT_TABLE table=test-table [id=83f9e4f263464aef8d7f3f7da3045937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:34.157159 847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet decc3d009ee048f2b36d091edcdf156e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:34.157714 957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.136637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44911 (local address 127.0.0.1:55062)
0806 01:05:34.148922 (+ 12285us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:34.148947 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:34.149081 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:34.150980 (+ 1899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:34.150984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:34.151011 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:34.151174 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.151186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.153238 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.153243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.155142 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.155158 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.156914 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.156920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.157094 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.157135 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 01:05:34.157252 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 01:05:34.157389 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12156,"thread_start_us":13548,"threads_started":1}
I20260806 01:05:34.157809 962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.137099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55062 (local address 127.5.108.130:44911)
0806 01:05:34.148388 (+ 11289us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.148401 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.149029 (+ 628us) server_negotiation.cc:407] Connection header received
0806 01:05:34.149257 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.149269 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.149577 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.151360 (+ 1783us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:34.151644 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.153083 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.155342 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.156743 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.156902 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.157099 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.157298 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.157456 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11102,"thread_start_us":10964,"threads_started":1}
I20260806 01:05:34.158977 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.138642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:45779 (local address 127.0.0.1:44610)
0806 01:05:34.150475 (+ 11833us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:34.150495 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:34.150627 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:34.152048 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:34.152053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:34.152078 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:34.152481 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.152494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.154243 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.154248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.156372 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.156387 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.158200 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.158207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.158376 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.158406 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:05:34.158502 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 01:05:34.158630 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11690,"thread_start_us":12073,"threads_started":1}
I20260806 01:05:34.159055 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.138919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.5.108.129:45779)
0806 01:05:34.151120 (+ 12201us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.151130 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.151157 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:05:34.151264 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.151274 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.151832 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.152036 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:34.152674 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.154061 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.156540 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.157980 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.158120 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.158342 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.158560 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.158734 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12039,"thread_start_us":11862,"threads_started":1}
I20260806 01:05:34.161554 650 tablet_service.cc:1467] Processing CreateTablet for tablet decc3d009ee048f2b36d091edcdf156e (DEFAULT_TABLE table=test-table [id=83f9e4f263464aef8d7f3f7da3045937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:34.162650 650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet decc3d009ee048f2b36d091edcdf156e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:34.162537 445 tablet_service.cc:1467] Processing CreateTablet for tablet decc3d009ee048f2b36d091edcdf156e (DEFAULT_TABLE table=test-table [id=83f9e4f263464aef8d7f3f7da3045937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:34.163893 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet decc3d009ee048f2b36d091edcdf156e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:34.172736 968 tablet_bootstrap.cc:492] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: Bootstrap starting.
I20260806 01:05:34.174531 969 tablet_bootstrap.cc:492] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: Bootstrap starting.
I20260806 01:05:34.177096 968 tablet_bootstrap.cc:654] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:34.178098 968 log.cc:824] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:34.179044 969 tablet_bootstrap.cc:654] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:34.180508 969 log.cc:824] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:34.182027 972 tablet_bootstrap.cc:492] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: Bootstrap starting.
I20260806 01:05:34.184501 969 tablet_bootstrap.cc:492] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: No bootstrap required, opened a new log
I20260806 01:05:34.184844 969 ts_tablet_manager.cc:1397] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260806 01:05:34.184937 972 tablet_bootstrap.cc:654] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:34.186072 972 log.cc:824] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:34.187361 968 tablet_bootstrap.cc:492] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: No bootstrap required, opened a new log
I20260806 01:05:34.187738 968 ts_tablet_manager.cc:1397] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260806 01:05:34.193109 972 tablet_bootstrap.cc:492] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: No bootstrap required, opened a new log
I20260806 01:05:34.193364 972 ts_tablet_manager.cc:1397] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.009s
I20260806 01:05:34.195684 969 raft_consensus.cc:348] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } }
I20260806 01:05:34.196092 969 raft_consensus.cc:374] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:34.196185 969 raft_consensus.cc:729] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e65bdcb2362498b8bb1c4c548290939, State: Initialized, Role: FOLLOWER
I20260806 01:05:34.196815 969 consensus_queue.cc:260] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } }
I20260806 01:05:34.197973 753 heartbeater.cc:503] Master 127.5.108.190:44435 was elected leader, sending a full tablet report...
I20260806 01:05:34.198239 968 raft_consensus.cc:348] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } }
I20260806 01:05:34.198684 969 ts_tablet_manager.cc:1428] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: Time spent starting tablet: real 0.014s user 0.004s sys 0.008s
I20260806 01:05:34.198748 968 raft_consensus.cc:374] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:34.198838 968 raft_consensus.cc:729] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca473b3e89ef4cd3bc3cb8475733fa3c, State: Initialized, Role: FOLLOWER
I20260806 01:05:34.199518 968 consensus_queue.cc:260] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } }
I20260806 01:05:34.202193 972 raft_consensus.cc:348] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } }
I20260806 01:05:34.202507 972 raft_consensus.cc:374] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:34.202682 972 raft_consensus.cc:729] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c9a2af7c7a44089a3a2a03260ee74c, State: Initialized, Role: FOLLOWER
I20260806 01:05:34.203254 972 consensus_queue.cc:260] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } }
I20260806 01:05:34.204105 940 heartbeater.cc:503] Master 127.5.108.190:44435 was elected leader, sending a full tablet report...
I20260806 01:05:34.204593 968 ts_tablet_manager.cc:1428] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
I20260806 01:05:34.209501 972 ts_tablet_manager.cc:1428] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
W20260806 01:05:34.298658 537 tablet.cc:2369] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:05:34.304226 756 tablet.cc:2369] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:05:34.318717 941 tablet.cc:2369] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:05:34.351352 978 raft_consensus.cc:482] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:34.351693 978 raft_consensus.cc:504] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } }
I20260806 01:05:34.353543 978 leader_election.cc:288] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e65bdcb2362498b8bb1c4c548290939 (127.5.108.130:44911), b9c9a2af7c7a44089a3a2a03260ee74c (127.5.108.129:45779)
I20260806 01:05:34.358917 962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.353993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:54593 (local address 127.5.108.130:44911)
0806 01:05:34.354203 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.354210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.354230 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:05:34.354313 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.354320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.354395 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.354516 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:34.354976 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.355780 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.357264 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.358015 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.358181 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.358269 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.358518 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.358641 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:05:34.358923 942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.353807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44911 (local address 127.5.108.131:54593)
0806 01:05:34.354032 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:34.354053 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:34.354202 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:34.354567 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:34.354590 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:34.354619 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:34.354830 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.354839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.355917 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.355922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.357100 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.357112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.358172 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.358178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.358362 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.358385 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 01:05:34.358492 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 01:05:34.358619 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 01:05:34.360073 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.353807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:45779 (local address 127.5.108.131:53203)
0806 01:05:34.354821 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:34.354835 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:34.354986 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:34.355317 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:34.355321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:34.355345 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:34.355699 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.355708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.356629 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.356640 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.357912 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:34.357923 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.359294 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.359301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:34.359501 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.359542 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 01:05:34.359617 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 01:05:34.359698 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":194,"thread_start_us":435,"threads_started":1}
I20260806 01:05:34.360145 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.353973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:53203 (local address 127.5.108.129:45779)
0806 01:05:34.354204 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.354212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.354928 (+ 716us) server_negotiation.cc:407] Connection header received
0806 01:05:34.355128 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.355131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.355190 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.355292 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:34.355870 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.356493 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.358070 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.359002 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.359180 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.359248 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.359676 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.359868 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:05:34.360083 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decc3d009ee048f2b36d091edcdf156e" candidate_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e65bdcb2362498b8bb1c4c548290939" is_pre_election: true
I20260806 01:05:34.360728 677 raft_consensus.cc:2454] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca473b3e89ef4cd3bc3cb8475733fa3c in term 0.
I20260806 01:05:34.361178 473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decc3d009ee048f2b36d091edcdf156e" candidate_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" is_pre_election: true
I20260806 01:05:34.361403 782 leader_election.cc:302] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e65bdcb2362498b8bb1c4c548290939, ca473b3e89ef4cd3bc3cb8475733fa3c; no voters:
I20260806 01:05:34.361689 473 raft_consensus.cc:2454] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca473b3e89ef4cd3bc3cb8475733fa3c in term 0.
I20260806 01:05:34.361783 978 raft_consensus.cc:2781] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:05:34.361937 978 raft_consensus.cc:482] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:05:34.362030 978 raft_consensus.cc:3037] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:34.364463 978 raft_consensus.cc:504] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } }
I20260806 01:05:34.365109 978 leader_election.cc:288] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [CANDIDATE]: Term 1 election: Requested vote from peers 5e65bdcb2362498b8bb1c4c548290939 (127.5.108.130:44911), b9c9a2af7c7a44089a3a2a03260ee74c (127.5.108.129:45779)
I20260806 01:05:34.365437 677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decc3d009ee048f2b36d091edcdf156e" candidate_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e65bdcb2362498b8bb1c4c548290939"
I20260806 01:05:34.365656 677 raft_consensus.cc:3037] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:34.365634 473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decc3d009ee048f2b36d091edcdf156e" candidate_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c"
I20260806 01:05:34.365816 473 raft_consensus.cc:3037] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:34.367859 677 raft_consensus.cc:2454] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca473b3e89ef4cd3bc3cb8475733fa3c in term 1.
I20260806 01:05:34.368124 473 raft_consensus.cc:2454] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca473b3e89ef4cd3bc3cb8475733fa3c in term 1.
I20260806 01:05:34.368286 782 leader_election.cc:302] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e65bdcb2362498b8bb1c4c548290939, ca473b3e89ef4cd3bc3cb8475733fa3c; no voters:
I20260806 01:05:34.368580 978 raft_consensus.cc:2781] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:34.369058 978 raft_consensus.cc:686] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 1 LEADER]: Becoming Leader. State: Replica: ca473b3e89ef4cd3bc3cb8475733fa3c, State: Running, Role: LEADER
I20260806 01:05:34.369598 978 consensus_queue.cc:237] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } }
I20260806 01:05:34.374287 32722 catalog_manager.cc:5495] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c reported cstate change: term changed from 0 to 1, leader changed from <none> to ca473b3e89ef4cd3bc3cb8475733fa3c (127.5.108.131). New cstate: current_term: 1 leader_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca473b3e89ef4cd3bc3cb8475733fa3c" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 38061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 } health_report { overall_health: UNKNOWN } } }
I20260806 01:05:34.441819 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.436502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.5.108.131:38061)
0806 01:05:34.436720 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.436725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.436758 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:05:34.436871 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.436875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.436958 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.437111 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:05:34.437505 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.438224 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.439633 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.440030 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.440164 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.440780 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:05:34.441227 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:05:34.441413 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.441540 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.441587 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:05:34.449327 473 raft_consensus.cc:1261] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [term 1 FOLLOWER]: Refusing update from remote peer ca473b3e89ef4cd3bc3cb8475733fa3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:34.449371 677 raft_consensus.cc:1261] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [term 1 FOLLOWER]: Refusing update from remote peer ca473b3e89ef4cd3bc3cb8475733fa3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:34.450184 985 consensus_queue.cc:1035] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c9a2af7c7a44089a3a2a03260ee74c" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 45779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:34.450500 978 consensus_queue.cc:1035] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e65bdcb2362498b8bb1c4c548290939" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:34.475533 32721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:34.475863 32721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:34.479329 32721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5426c60fad0442609e869bd91c22f01d: Sending DeleteTablet for 3 replicas of tablet decc3d009ee048f2b36d091edcdf156e
I20260806 01:05:34.480624 650 tablet_service.cc:1514] Processing DeleteTablet for tablet decc3d009ee048f2b36d091edcdf156e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:34 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:55062
I20260806 01:05:34.480655 847 tablet_service.cc:1514] Processing DeleteTablet for tablet decc3d009ee048f2b36d091edcdf156e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:34 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:38708
I20260806 01:05:34.481330 445 tablet_service.cc:1514] Processing DeleteTablet for tablet decc3d009ee048f2b36d091edcdf156e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:34 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:44610
I20260806 01:05:34.481469 1002 tablet_replica.cc:330] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: stopping tablet replica
I20260806 01:05:34.481848 1002 raft_consensus.cc:2229] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:05:34.482442 1002 raft_consensus.cc:2258] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:34.484485 1002 ts_tablet_manager.cc:1905] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:34.487352 1003 tablet_replica.cc:330] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: stopping tablet replica
I20260806 01:05:34.487793 1003 raft_consensus.cc:2229] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:34.487915 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 344
I20260806 01:05:34.490520 1003 raft_consensus.cc:2258] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:34.493889 1002 ts_tablet_manager.cc:1918] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:34.494130 1002 log.cc:1196] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/decc3d009ee048f2b36d091edcdf156e
I20260806 01:05:34.494943 1002 ts_tablet_manager.cc:1939] T decc3d009ee048f2b36d091edcdf156e P ca473b3e89ef4cd3bc3cb8475733fa3c: Deleting consensus metadata
I20260806 01:05:34.496666 32696 catalog_manager.cc:4841] TS ca473b3e89ef4cd3bc3cb8475733fa3c (127.5.108.131:38061): tablet decc3d009ee048f2b36d091edcdf156e (table test-table [id=83f9e4f263464aef8d7f3f7da3045937]) successfully deleted
I20260806 01:05:34.496750 1003 ts_tablet_manager.cc:1905] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:34.496948 1004 tablet_replica.cc:330] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: stopping tablet replica
I20260806 01:05:34.497937 1004 raft_consensus.cc:2229] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:34.498463 1004 raft_consensus.cc:2258] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:34.500512 1004 ts_tablet_manager.cc:1905] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:34.503985 1003 ts_tablet_manager.cc:1918] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:34.504590 1003 log.cc:1196] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/decc3d009ee048f2b36d091edcdf156e
I20260806 01:05:34.505252 1003 ts_tablet_manager.cc:1939] T decc3d009ee048f2b36d091edcdf156e P 5e65bdcb2362498b8bb1c4c548290939: Deleting consensus metadata
I20260806 01:05:34.507068 32697 catalog_manager.cc:4841] TS 5e65bdcb2362498b8bb1c4c548290939 (127.5.108.130:44911): tablet decc3d009ee048f2b36d091edcdf156e (table test-table [id=83f9e4f263464aef8d7f3f7da3045937]) successfully deleted
I20260806 01:05:34.507483 1004 ts_tablet_manager.cc:1918] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:34.507651 1004 log.cc:1196] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/decc3d009ee048f2b36d091edcdf156e
Aug 06 01:05:34 dist-test-slave-x4qh krb5kdc[32649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:05:34.508198 1004 ts_tablet_manager.cc:1939] T decc3d009ee048f2b36d091edcdf156e P b9c9a2af7c7a44089a3a2a03260ee74c: Deleting consensus metadata
I20260806 01:05:34.509886 32698 catalog_manager.cc:4841] TS b9c9a2af7c7a44089a3a2a03260ee74c (127.5.108.129:45779): tablet decc3d009ee048f2b36d091edcdf156e (table test-table [id=83f9e4f263464aef8d7f3f7da3045937]) successfully deleted
I20260806 01:05:34.515108 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.489161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44616 (local address 127.5.108.129:45779)
0806 01:05:34.489327 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.489333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.497493 (+ 8160us) server_negotiation.cc:407] Connection header received
0806 01:05:34.497775 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.497782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.497859 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.497935 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:34.499874 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.500931 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.502743 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.503325 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.503462 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.509088 (+ 5626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:34.509101 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:34.509117 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:34.509173 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:34.512489 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.513159 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.513166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.513168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.513301 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.513712 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.513715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.513717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.514023 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:34.514324 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.514598 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.514789 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 01:05:34.515811 528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:34.625483 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 344
I20260806 01:05:34.651315 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 547
Aug 06 01:05:34 dist-test-slave-x4qh krb5kdc[32649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:05:34.667363 962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.652455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.5.108.130:44911)
0806 01:05:34.652672 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.652679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.652704 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:05:34.652757 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.652760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.652819 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.652934 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:34.654274 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.655087 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.656419 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.656821 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.656955 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.661580 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:34.661594 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:34.661610 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:34.661644 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:34.664622 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.665275 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.665283 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.665285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.665405 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.665811 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.665816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.665819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.666183 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:34.666500 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.666817 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.667038 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:05:34.668102 742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:34.756937 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 547
I20260806 01:05:34.785354 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 760
Aug 06 01:05:34 dist-test-slave-x4qh krb5kdc[32649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:05:34.801366 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:34.786122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38730 (local address 127.5.108.131:38061)
0806 01:05:34.786317 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:34.786323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:34.786412 (+ 89us) server_negotiation.cc:407] Connection header received
0806 01:05:34.786610 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:34.786616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:34.786689 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:34.786804 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:34.788292 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.789156 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.790357 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:34.790845 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:34.790987 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:34.795462 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:34.795474 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:34.795489 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:34.795519 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:34.798417 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.799120 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.799141 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.799144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.799292 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:34.799859 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:34.799868 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:34.799873 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:34.800224 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:34.800574 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:34.800832 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 01:05:34.801038 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 01:05:34.801980 933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:34.918557 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 760
I20260806 01:05:34.945614 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 32670
I20260806 01:05:34.946614 302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:35.053575 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 32670
2026-08-06T01:05:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4622 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T01:05:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:35Z Disabled control of system clock
I20260806 01:05:35.146088 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37899
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:37899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:05:35.474159 1017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:35.474391 1017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:35.474469 1017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:35.481339 1017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:35.481428 1017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:35.481463 1017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:35.481515 1017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:35.481556 1017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:35.490130 1017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:37899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:35.491192 1017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:35.492784 1017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:35.504914 1024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:35.504957 1026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:35.504810 1028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:35.505916 1017 server_base.cc:1034] running on GCE node
I20260806 01:05:35.506709 1017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:35.508203 1017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:35.509436 1017 hybrid_clock.cc:648] HybridClock initialized: now 1785978335509379 us; error 43 us; skew 500 ppm
I20260806 01:05:35.509881 1017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:35.513396 1017 webserver.cc:460] Webserver started at http://127.5.108.190:44231/ using document root <none> and password file <none>
I20260806 01:05:35.514349 1017 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:35.514472 1017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:35.514869 1017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:35.517892 1017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "af9f89a085c64fd6908b74cde15c8606"
format_stamp: "Formatted at 2026-08-06 01:05:35 on dist-test-slave-x4qh"
server_key: "a1a53ded361d713dfbdb6d14cebafb1d"
server_key_iv: "acb17147b709e37da00a5b4b5ca342e1"
server_key_version: "encryptionkey@0"
I20260806 01:05:35.518801 1017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "af9f89a085c64fd6908b74cde15c8606"
format_stamp: "Formatted at 2026-08-06 01:05:35 on dist-test-slave-x4qh"
server_key: "a1a53ded361d713dfbdb6d14cebafb1d"
server_key_iv: "acb17147b709e37da00a5b4b5ca342e1"
server_key_version: "encryptionkey@0"
I20260806 01:05:35.523810 1017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:05:35.527076 1040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:35.528436 1017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:35.528618 1017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "af9f89a085c64fd6908b74cde15c8606"
format_stamp: "Formatted at 2026-08-06 01:05:35 on dist-test-slave-x4qh"
server_key: "a1a53ded361d713dfbdb6d14cebafb1d"
server_key_iv: "acb17147b709e37da00a5b4b5ca342e1"
server_key_version: "encryptionkey@0"
I20260806 01:05:35.528831 1017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:35.556219 1017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:35.556978 1017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:35.557261 1017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:35.575701 1017 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:37899
I20260806 01:05:35.575732 1122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:37899 every 8 connection(s)
I20260806 01:05:35.576896 1017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:35.579159 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 1017
I20260806 01:05:35.579355 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:35.579877 5554 external_mini_cluster.cc:1428] Setting key 8b8f17c71c375b17d1f1473ee490d137
I20260806 01:05:35.580680 1124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:35.594837 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:35.581702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.5.108.190:37899)
0806 01:05:35.582785 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:35.582800 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:35.582850 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:05:35.585373 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:35.585399 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:35.585943 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:35.586398 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:35.586907 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:35.588967 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:35.590263 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:35.591221 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:35.591413 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:35.592420 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:35.592429 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:35.592443 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:35.593094 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:35.593192 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:35.593220 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:35.593331 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:35.593465 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 01:05:35.593725 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":372,"threads_started":1}
I20260806 01:05:35.595265 1124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606: Bootstrap starting.
I20260806 01:05:35.598228 1124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:35.599221 1124 log.cc:824] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:35.601754 1124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606: No bootstrap required, opened a new log
I20260806 01:05:35.608635 1124 raft_consensus.cc:348] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9f89a085c64fd6908b74cde15c8606" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37899 } }
I20260806 01:05:35.608978 1124 raft_consensus.cc:374] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:35.609112 1124 raft_consensus.cc:729] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af9f89a085c64fd6908b74cde15c8606, State: Initialized, Role: FOLLOWER
I20260806 01:05:35.609701 1124 consensus_queue.cc:260] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9f89a085c64fd6908b74cde15c8606" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37899 } }
I20260806 01:05:35.609925 1124 raft_consensus.cc:388] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:35.610064 1124 raft_consensus.cc:482] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:35.610229 1124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:35.612327 1124 raft_consensus.cc:504] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9f89a085c64fd6908b74cde15c8606" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37899 } }
I20260806 01:05:35.612869 1124 leader_election.cc:302] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af9f89a085c64fd6908b74cde15c8606; no voters:
I20260806 01:05:35.613374 1124 leader_election.cc:288] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:35.613507 1134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:35.614272 1134 raft_consensus.cc:686] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [term 1 LEADER]: Becoming Leader. State: Replica: af9f89a085c64fd6908b74cde15c8606, State: Running, Role: LEADER
I20260806 01:05:35.614785 1134 consensus_queue.cc:237] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9f89a085c64fd6908b74cde15c8606" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37899 } }
I20260806 01:05:35.615187 1124 sys_catalog.cc:564] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:35.617388 1138 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af9f89a085c64fd6908b74cde15c8606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9f89a085c64fd6908b74cde15c8606" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37899 } } }
I20260806 01:05:35.617800 1138 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:35.617610 1139 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af9f89a085c64fd6908b74cde15c8606. Latest consensus state: current_term: 1 leader_uuid: "af9f89a085c64fd6908b74cde15c8606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9f89a085c64fd6908b74cde15c8606" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37899 } } }
I20260806 01:05:35.618046 1139 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:35.620230 1145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:35.625344 1145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:35.633745 1145 catalog_manager.cc:1347] Generated new cluster ID: 861206665a624f5bafc1681efae7b920
I20260806 01:05:35.633865 1145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:35.645041 1145 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:35.646158 1145 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:35.660135 1145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606: Generated new TSK 0
I20260806 01:05:35.660848 1145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:05:35.665926 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37899
--builtin_ntp_servers=127.5.108.148:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:05:35.998298 1163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:35.998553 1163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:35.998699 1163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:36.005213 1163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:36.005295 1163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:36.005497 1163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:05:36.013768 1163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:36.014918 1163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:36.016757 1163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:36.029726 1169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:36.029675 1174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:36.029788 1170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:36.030768 1163 server_base.cc:1034] running on GCE node
I20260806 01:05:36.031453 1163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:36.032321 1163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:36.033572 1163 hybrid_clock.cc:648] HybridClock initialized: now 1785978336033510 us; error 44 us; skew 500 ppm
I20260806 01:05:36.033910 1163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:36.036053 1163 webserver.cc:460] Webserver started at http://127.5.108.129:38255/ using document root <none> and password file <none>
I20260806 01:05:36.036764 1163 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:36.036900 1163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:36.037218 1163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:36.040081 1163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8"
format_stamp: "Formatted at 2026-08-06 01:05:36 on dist-test-slave-x4qh"
server_key: "6de773fcc678d2df407823afd0bff811"
server_key_iv: "33992ae08ecb74dbf09665730722bdfe"
server_key_version: "encryptionkey@0"
I20260806 01:05:36.040874 1163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8"
format_stamp: "Formatted at 2026-08-06 01:05:36 on dist-test-slave-x4qh"
server_key: "6de773fcc678d2df407823afd0bff811"
server_key_iv: "33992ae08ecb74dbf09665730722bdfe"
server_key_version: "encryptionkey@0"
I20260806 01:05:36.045701 1163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:05:36.048889 1181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:36.050016 1163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 01:05:36.050181 1163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8"
format_stamp: "Formatted at 2026-08-06 01:05:36 on dist-test-slave-x4qh"
server_key: "6de773fcc678d2df407823afd0bff811"
server_key_iv: "33992ae08ecb74dbf09665730722bdfe"
server_key_version: "encryptionkey@0"
I20260806 01:05:36.050376 1163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:36.067646 1163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:36.068396 1163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:36.068711 1163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:36.069595 1163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:36.070946 1163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:36.071072 1163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:36.071221 1163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:36.071305 1163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:36.104521 1163 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:44473
I20260806 01:05:36.104604 1353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:44473 every 8 connection(s)
I20260806 01:05:36.105821 1163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:05:36.107542 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 1163
I20260806 01:05:36.107718 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:05:36.108182 5554 external_mini_cluster.cc:1428] Setting key 47cd59d6ec52f8f56a520985fa95d23b
I20260806 01:05:36.111104 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37899
--builtin_ntp_servers=127.5.108.148:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:05:36.117815 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:36.108540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:39669 (local address 127.5.108.190:37899)
0806 01:05:36.109028 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:36.109034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:36.110260 (+ 1226us) server_negotiation.cc:407] Connection header received
0806 01:05:36.112020 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:36.112025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:36.112103 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:36.112224 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:36.113179 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:36.113851 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:36.115201 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:36.115689 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:36.115828 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:36.117027 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:36.117033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:36.117039 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:36.117060 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:36.117106 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:36.117116 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:36.117207 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:36.117407 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 01:05:36.117493 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":240,"threads_started":1}
I20260806 01:05:36.118794 1358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:36.109144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37899 (local address 127.5.108.129:39669)
0806 01:05:36.110087 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:36.110142 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:36.111785 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:36.112444 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:36.112451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:36.112522 (+ 71us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:36.113027 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:36.113042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:36.114006 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:36.114011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:36.115062 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:36.115073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:36.115891 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:36.115899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:36.116114 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:36.116719 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:05:36.116721 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:36.116760 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:05:36.116846 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:36.117208 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:36.117221 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:36.117230 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 01:05:36.117395 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 01:05:36.117718 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":300,"threads_started":1}
I20260806 01:05:36.120425 1354 heartbeater.cc:347] Connected to a master server at 127.5.108.190:37899
I20260806 01:05:36.120710 1354 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:36.121387 1354 heartbeater.cc:511] Master 127.5.108.190:37899 requested a full tablet report, sending...
I20260806 01:05:36.123450 1063 ts_manager.cc:194] Registered new tserver with Master: f9afd1b7df3f402e9f4b36c51cdeffe8 (127.5.108.129:44473)
I20260806 01:05:36.125094 1063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.129:39669
W20260806 01:05:36.437201 1359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:36.437438 1359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:36.437589 1359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:36.444286 1359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:36.444388 1359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:36.444576 1359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:05:36.453094 1359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:36.454293 1359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:36.455977 1359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:36.468920 1365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:36.468954 1368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:36.468901 1364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:36.469621 1359 server_base.cc:1034] running on GCE node
I20260806 01:05:36.470111 1359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:36.470944 1359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:36.472142 1359 hybrid_clock.cc:648] HybridClock initialized: now 1785978336472087 us; error 44 us; skew 500 ppm
I20260806 01:05:36.472435 1359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:36.474381 1359 webserver.cc:460] Webserver started at http://127.5.108.130:40755/ using document root <none> and password file <none>
I20260806 01:05:36.474992 1359 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:36.475102 1359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:36.475479 1359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:36.478034 1359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b"
format_stamp: "Formatted at 2026-08-06 01:05:36 on dist-test-slave-x4qh"
server_key: "995ad45e4112184e4064681981ab8b4c"
server_key_iv: "0ece6b7b0ee8b36a11ee901dc5277493"
server_key_version: "encryptionkey@0"
I20260806 01:05:36.478832 1359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b"
format_stamp: "Formatted at 2026-08-06 01:05:36 on dist-test-slave-x4qh"
server_key: "995ad45e4112184e4064681981ab8b4c"
server_key_iv: "0ece6b7b0ee8b36a11ee901dc5277493"
server_key_version: "encryptionkey@0"
I20260806 01:05:36.483564 1359 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260806 01:05:36.486642 1377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:36.487877 1359 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 01:05:36.488049 1359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b"
format_stamp: "Formatted at 2026-08-06 01:05:36 on dist-test-slave-x4qh"
server_key: "995ad45e4112184e4064681981ab8b4c"
server_key_iv: "0ece6b7b0ee8b36a11ee901dc5277493"
server_key_version: "encryptionkey@0"
I20260806 01:05:36.488270 1359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:36.521898 1359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:36.522679 1359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:36.522969 1359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:36.523911 1359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:36.525321 1359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:36.525445 1359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:36.525575 1359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:36.525674 1359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:36.557580 1359 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:46275
I20260806 01:05:36.557662 1536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:46275 every 8 connection(s)
I20260806 01:05:36.558836 1359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:05:36.562951 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 1359
I20260806 01:05:36.563125 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:05:36.563679 5554 external_mini_cluster.cc:1428] Setting key b370fe746b3832646a4e4233ab81a166
I20260806 01:05:36.566529 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37899
--builtin_ntp_servers=127.5.108.148:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:05:36.571810 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:36.561205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:48173 (local address 127.5.108.190:37899)
0806 01:05:36.561377 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:36.561381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:36.562565 (+ 1184us) server_negotiation.cc:407] Connection header received
0806 01:05:36.564771 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:36.564776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:36.564852 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:36.564972 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:36.565987 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:36.566692 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:36.569177 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:36.569571 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:36.569664 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:36.571116 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:36.571121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:36.571124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:36.571147 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:36.571178 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:36.571188 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:36.571255 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:36.571512 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 01:05:36.571584 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":54}
I20260806 01:05:36.572966 1540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:36.561635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37899 (local address 127.5.108.130:48173)
0806 01:05:36.562417 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:36.562448 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:36.564556 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:36.565135 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:36.565142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:36.565214 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:36.565810 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:36.565833 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:36.566861 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:36.566868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:36.569009 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:36.569027 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:36.569777 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:36.569786 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:36.570095 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:36.570794 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:05:36.570799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:36.570845 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:05:36.570934 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:36.571284 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:36.571295 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:36.571304 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 01:05:36.571492 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 01:05:36.571840 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":242,"threads_started":1}
I20260806 01:05:36.574345 1538 heartbeater.cc:347] Connected to a master server at 127.5.108.190:37899
I20260806 01:05:36.574656 1538 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:36.575615 1538 heartbeater.cc:511] Master 127.5.108.190:37899 requested a full tablet report, sending...
I20260806 01:05:36.577188 1063 ts_manager.cc:194] Registered new tserver with Master: 2bef9d2b8ae84f8d9ebb35fbf2a7192b (127.5.108.130:46275)
I20260806 01:05:36.578263 1063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.130:48173
W20260806 01:05:36.909211 1541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:36.909468 1541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:36.909633 1541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:36.917075 1541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:36.917171 1541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:36.917371 1541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:05:36.926069 1541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:37899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:36.927258 1541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:36.929068 1541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:36.941900 1551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:36.941962 1546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:36.941994 1547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:36.945618 1541 server_base.cc:1034] running on GCE node
I20260806 01:05:36.946259 1541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:36.947320 1541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:36.948567 1541 hybrid_clock.cc:648] HybridClock initialized: now 1785978336948525 us; error 37 us; skew 500 ppm
I20260806 01:05:36.948880 1541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:36.950953 1541 webserver.cc:460] Webserver started at http://127.5.108.131:33463/ using document root <none> and password file <none>
I20260806 01:05:36.951715 1541 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:36.951826 1541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:36.952129 1541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:36.954972 1541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf"
format_stamp: "Formatted at 2026-08-06 01:05:36 on dist-test-slave-x4qh"
server_key: "489afd4a4458bc6977179adc3b68f5b8"
server_key_iv: "21884efe223368ab2914eb70fb85dd66"
server_key_version: "encryptionkey@0"
I20260806 01:05:36.955734 1541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf"
format_stamp: "Formatted at 2026-08-06 01:05:36 on dist-test-slave-x4qh"
server_key: "489afd4a4458bc6977179adc3b68f5b8"
server_key_iv: "21884efe223368ab2914eb70fb85dd66"
server_key_version: "encryptionkey@0"
I20260806 01:05:36.960415 1541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:05:36.963646 1557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:36.964864 1541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:05:36.965021 1541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf"
format_stamp: "Formatted at 2026-08-06 01:05:36 on dist-test-slave-x4qh"
server_key: "489afd4a4458bc6977179adc3b68f5b8"
server_key_iv: "21884efe223368ab2914eb70fb85dd66"
server_key_version: "encryptionkey@0"
I20260806 01:05:36.965226 1541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:36.980304 1541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:36.980993 1541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:36.981294 1541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:36.982175 1541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:36.983690 1541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:36.983799 1541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:36.983937 1541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:36.984018 1541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:37.015976 1541 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:42711
I20260806 01:05:37.016083 1713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:42711 every 8 connection(s)
I20260806 01:05:37.017342 1541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:05:37.024534 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 1541
I20260806 01:05:37.024755 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:05:37.025180 5554 external_mini_cluster.cc:1428] Setting key 62b0d7606e7296435d3db0f61142df92
I20260806 01:05:37.030005 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.019927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:57821 (local address 127.5.108.190:37899)
0806 01:05:37.020106 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.020110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.021447 (+ 1337us) server_negotiation.cc:407] Connection header received
0806 01:05:37.023138 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.023142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.023205 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.023297 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:37.024423 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.025322 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.027465 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.027943 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.028059 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.029379 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:37.029383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:37.029386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:37.029402 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:37.029432 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:37.029443 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:37.029512 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.029704 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.029776 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:05:37.030947 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.020463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:37899 (local address 127.5.108.131:57821)
0806 01:05:37.021294 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:37.021334 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:37.022908 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:37.023543 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:37.023550 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:37.023625 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:37.024211 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.024228 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.025479 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.025491 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.027308 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.027322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.028191 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.028197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.028390 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.029093 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:05:37.029095 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:37.029133 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:05:37.029224 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:37.029528 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:37.029536 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:37.029541 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 01:05:37.029694 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 01:05:37.029948 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":265,"threads_started":1}
I20260806 01:05:37.032163 1714 heartbeater.cc:347] Connected to a master server at 127.5.108.190:37899
I20260806 01:05:37.032495 1714 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:37.033221 1714 heartbeater.cc:511] Master 127.5.108.190:37899 requested a full tablet report, sending...
I20260806 01:05:37.034512 1063 ts_manager.cc:194] Registered new tserver with Master: 0fdb17178ff14fa98ef5397a3f3b9ebf (127.5.108.131:42711)
I20260806 01:05:37.035271 1063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.131:57821
I20260806 01:05:37.040215 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:05:37.049748 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.044028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.5.108.190:37899)
0806 01:05:37.044199 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.044206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.044606 (+ 400us) server_negotiation.cc:407] Connection header received
0806 01:05:37.044805 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.044812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.044883 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.044982 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:37.045481 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.046102 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.047357 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.047781 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.047937 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.049048 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:37.049053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:37.049056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:37.049071 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:37.049111 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:37.049121 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:37.049268 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.049455 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.049506 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 01:05:37.053194 1063 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:05:37.060101 1063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:05:37.084223 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.071555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54588 (local address 127.5.108.129:44473)
0806 01:05:37.072110 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.072121 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.073571 (+ 1450us) server_negotiation.cc:407] Connection header received
0806 01:05:37.074016 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.074024 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.074260 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.074420 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:37.075617 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.077493 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.081190 (+ 3697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.082695 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.082803 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.083062 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.083824 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.083975 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":305,"threads_started":1}
I20260806 01:05:37.084256 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.071357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44473 (local address 127.0.0.1:54588)
0806 01:05:37.073418 (+ 2061us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:37.073464 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:37.073731 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:37.074801 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:37.074810 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:37.074848 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:37.075354 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.075393 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.077725 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.077733 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.080939 (+ 3206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.080966 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.083244 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.083253 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.083505 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.083564 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 01:05:37.083749 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 01:05:37.083929 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1830,"thread_start_us":10051,"threads_started":1}
I20260806 01:05:37.088397 1251 tablet_service.cc:1467] Processing CreateTablet for tablet d05dd04fa5c246ac9b875f85d334c248 (DEFAULT_TABLE table=test-table [id=aa3a3b334b244b5c8f51b0670bc1609f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:37.089637 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.072237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.5.108.130:46275)
0806 01:05:37.072900 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.072910 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.081699 (+ 8789us) server_negotiation.cc:407] Connection header received
0806 01:05:37.082028 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.082035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.082270 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.082432 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:37.083316 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.084851 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.087110 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.088099 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.088246 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.088482 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.089150 (+ 668us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.089319 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":1645,"threads_started":1}
I20260806 01:05:37.090052 1251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05dd04fa5c246ac9b875f85d334c248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:37.090551 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.071569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:46275 (local address 127.0.0.1:47522)
0806 01:05:37.081584 (+ 10015us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:37.081601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:37.081817 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:37.082701 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:37.082705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:37.082731 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:37.083128 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.083140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.085013 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.085018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.086922 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.086935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.088656 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.088674 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.088898 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.088934 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:05:37.089989 (+ 1055us) client_negotiation.cc:240] Negotiation successful
0806 01:05:37.090148 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9866,"mutex_wait_us":92,"thread_start_us":10252,"threads_started":1}
I20260806 01:05:37.090909 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.072695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:42711 (local address 127.0.0.1:41606)
0806 01:05:37.082063 (+ 9368us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:37.082078 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:37.082223 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:37.083058 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:37.083062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:37.083124 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:37.083571 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.083581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.086343 (+ 2762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.086348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.088375 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.088393 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.090205 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.090211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.090379 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.090413 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:05:37.090506 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 01:05:37.090623 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9263,"thread_start_us":1334,"threads_started":1}
I20260806 01:05:37.091194 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.073356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.5.108.131:42711)
0806 01:05:37.081392 (+ 8036us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.081403 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.082359 (+ 956us) server_negotiation.cc:407] Connection header received
0806 01:05:37.082469 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.082477 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.082771 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.082970 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:37.084608 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.086172 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.088596 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.089992 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.090122 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.090368 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.090600 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.090790 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7823,"thread_start_us":7651,"threads_started":1}
I20260806 01:05:37.094065 1443 tablet_service.cc:1467] Processing CreateTablet for tablet d05dd04fa5c246ac9b875f85d334c248 (DEFAULT_TABLE table=test-table [id=aa3a3b334b244b5c8f51b0670bc1609f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:37.094538 1630 tablet_service.cc:1467] Processing CreateTablet for tablet d05dd04fa5c246ac9b875f85d334c248 (DEFAULT_TABLE table=test-table [id=aa3a3b334b244b5c8f51b0670bc1609f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:37.095310 1443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05dd04fa5c246ac9b875f85d334c248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:37.095918 1630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05dd04fa5c246ac9b875f85d334c248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:37.112848 1739 tablet_bootstrap.cc:492] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Bootstrap starting.
I20260806 01:05:37.113626 1737 tablet_bootstrap.cc:492] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: Bootstrap starting.
I20260806 01:05:37.117890 1737 tablet_bootstrap.cc:654] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:37.117887 1739 tablet_bootstrap.cc:654] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:37.120059 1737 log.cc:824] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:37.120198 1739 log.cc:824] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:37.120718 1742 tablet_bootstrap.cc:492] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: Bootstrap starting.
I20260806 01:05:37.123013 1739 tablet_bootstrap.cc:492] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: No bootstrap required, opened a new log
I20260806 01:05:37.123294 1739 ts_tablet_manager.cc:1397] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 01:05:37.124099 1742 tablet_bootstrap.cc:654] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:37.125006 1742 log.cc:824] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:37.125463 1737 tablet_bootstrap.cc:492] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: No bootstrap required, opened a new log
I20260806 01:05:37.125797 1737 ts_tablet_manager.cc:1397] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260806 01:05:37.127478 1742 tablet_bootstrap.cc:492] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: No bootstrap required, opened a new log
I20260806 01:05:37.127763 1742 ts_tablet_manager.cc:1397] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 01:05:37.127794 1354 heartbeater.cc:503] Master 127.5.108.190:37899 was elected leader, sending a full tablet report...
I20260806 01:05:37.132795 1739 raft_consensus.cc:348] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } }
I20260806 01:05:37.133147 1739 raft_consensus.cc:374] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:37.133272 1739 raft_consensus.cc:729] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bef9d2b8ae84f8d9ebb35fbf2a7192b, State: Initialized, Role: FOLLOWER
I20260806 01:05:37.133772 1739 consensus_queue.cc:260] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } }
I20260806 01:05:37.134861 1538 heartbeater.cc:503] Master 127.5.108.190:37899 was elected leader, sending a full tablet report...
I20260806 01:05:37.135578 1739 ts_tablet_manager.cc:1428] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260806 01:05:37.136466 1737 raft_consensus.cc:348] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } }
I20260806 01:05:37.136889 1737 raft_consensus.cc:374] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:37.136983 1737 raft_consensus.cc:729] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9afd1b7df3f402e9f4b36c51cdeffe8, State: Initialized, Role: FOLLOWER
I20260806 01:05:37.137487 1737 consensus_queue.cc:260] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } }
I20260806 01:05:37.137699 1742 raft_consensus.cc:348] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } }
I20260806 01:05:37.138186 1742 raft_consensus.cc:374] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:37.138361 1742 raft_consensus.cc:729] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fdb17178ff14fa98ef5397a3f3b9ebf, State: Initialized, Role: FOLLOWER
I20260806 01:05:37.139233 1742 consensus_queue.cc:260] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } }
I20260806 01:05:37.140375 1714 heartbeater.cc:503] Master 127.5.108.190:37899 was elected leader, sending a full tablet report...
I20260806 01:05:37.140655 1737 ts_tablet_manager.cc:1428] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260806 01:05:37.141755 1742 ts_tablet_manager.cc:1428] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260806 01:05:37.160075 1746 raft_consensus.cc:482] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:37.160336 1746 raft_consensus.cc:504] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } }
I20260806 01:05:37.161698 1746 leader_election.cc:288] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9afd1b7df3f402e9f4b36c51cdeffe8 (127.5.108.129:44473), 0fdb17178ff14fa98ef5397a3f3b9ebf (127.5.108.131:42711)
I20260806 01:05:37.168833 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.162107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:33161 (local address 127.5.108.131:42711)
0806 01:05:37.162292 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.162301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.162943 (+ 642us) server_negotiation.cc:407] Connection header received
0806 01:05:37.163153 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.163157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.163206 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.163296 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:37.163907 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.164593 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.166494 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.167653 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.167819 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.167904 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.168442 (+ 538us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.168574 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:05:37.168833 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.162075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:58883 (local address 127.5.108.129:44473)
0806 01:05:37.162302 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.162310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.163034 (+ 724us) server_negotiation.cc:407] Connection header received
0806 01:05:37.163191 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.163194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.163243 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.163333 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:37.163961 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.164606 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.166610 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.167734 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.167865 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.167946 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.168442 (+ 496us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.168576 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:05:37.168929 1752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.162015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:44473 (local address 127.5.108.130:58883)
0806 01:05:37.162935 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:37.162952 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:37.163048 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:37.163346 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:37.163350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:37.163386 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:37.163818 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.163827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.164737 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.164744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.166359 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.166376 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.167886 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.167890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.168178 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.168257 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 01:05:37.168351 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 01:05:37.168503 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":182,"thread_start_us":466,"threads_started":1}
I20260806 01:05:37.168941 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.161945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:42711 (local address 127.5.108.130:33161)
0806 01:05:37.162839 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:37.162853 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:37.163015 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:37.163311 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:37.163315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:37.163337 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:37.163771 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.163782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.164721 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.164728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.166320 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:37.166331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.167851 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.167859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:37.168176 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.168254 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 01:05:37.168351 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 01:05:37.168595 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":174,"thread_start_us":636,"threads_started":1}
I20260806 01:05:37.169886 1283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05dd04fa5c246ac9b875f85d334c248" candidate_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" is_pre_election: true
I20260806 01:05:37.170373 1283 raft_consensus.cc:2454] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bef9d2b8ae84f8d9ebb35fbf2a7192b in term 0.
I20260806 01:05:37.170348 1659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05dd04fa5c246ac9b875f85d334c248" candidate_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" is_pre_election: true
I20260806 01:05:37.170867 1659 raft_consensus.cc:2454] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bef9d2b8ae84f8d9ebb35fbf2a7192b in term 0.
I20260806 01:05:37.171000 1380 leader_election.cc:302] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bef9d2b8ae84f8d9ebb35fbf2a7192b, f9afd1b7df3f402e9f4b36c51cdeffe8; no voters:
I20260806 01:05:37.171438 1746 raft_consensus.cc:2781] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:05:37.171612 1746 raft_consensus.cc:482] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:05:37.171744 1746 raft_consensus.cc:3037] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:37.173882 1746 raft_consensus.cc:504] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } }
I20260806 01:05:37.174444 1746 leader_election.cc:288] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [CANDIDATE]: Term 1 election: Requested vote from peers f9afd1b7df3f402e9f4b36c51cdeffe8 (127.5.108.129:44473), 0fdb17178ff14fa98ef5397a3f3b9ebf (127.5.108.131:42711)
I20260806 01:05:37.174891 1283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05dd04fa5c246ac9b875f85d334c248" candidate_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8"
I20260806 01:05:37.174899 1659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05dd04fa5c246ac9b875f85d334c248" candidate_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf"
I20260806 01:05:37.175087 1283 raft_consensus.cc:3037] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:37.175091 1659 raft_consensus.cc:3037] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:37.177250 1283 raft_consensus.cc:2454] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bef9d2b8ae84f8d9ebb35fbf2a7192b in term 1.
I20260806 01:05:37.177290 1659 raft_consensus.cc:2454] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bef9d2b8ae84f8d9ebb35fbf2a7192b in term 1.
I20260806 01:05:37.177627 1380 leader_election.cc:302] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bef9d2b8ae84f8d9ebb35fbf2a7192b, f9afd1b7df3f402e9f4b36c51cdeffe8; no voters:
I20260806 01:05:37.177937 1746 raft_consensus.cc:2781] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:37.178396 1746 raft_consensus.cc:686] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 1 LEADER]: Becoming Leader. State: Replica: 2bef9d2b8ae84f8d9ebb35fbf2a7192b, State: Running, Role: LEADER
I20260806 01:05:37.178836 1746 consensus_queue.cc:237] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } }
I20260806 01:05:37.183290 1063 catalog_manager.cc:5495] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bef9d2b8ae84f8d9ebb35fbf2a7192b (127.5.108.130). New cstate: current_term: 1 leader_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bef9d2b8ae84f8d9ebb35fbf2a7192b" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 } health_report { overall_health: UNKNOWN } } }
I20260806 01:05:37.197332 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.192481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.5.108.130:46275)
0806 01:05:37.192640 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.192644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.192735 (+ 91us) server_negotiation.cc:407] Connection header received
0806 01:05:37.192893 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.192896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.192970 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.193082 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:05:37.193387 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.194083 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.195501 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.195884 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.195979 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.196388 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:05:37.196851 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:05:37.196978 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.197081 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.197113 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 01:05:37.204613 1283 raft_consensus.cc:1261] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [term 1 FOLLOWER]: Refusing update from remote peer 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:37.204615 1659 raft_consensus.cc:1261] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [term 1 FOLLOWER]: Refusing update from remote peer 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:37.205482 1755 consensus_queue.cc:1035] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9afd1b7df3f402e9f4b36c51cdeffe8" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 44473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:37.205775 1746 consensus_queue.cc:1035] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fdb17178ff14fa98ef5397a3f3b9ebf" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 42711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:37.218008 1758 mvcc.cc:205] Tried to move back new op lower bound from 7315367269181108224 to 7315367269088608256. Current Snapshot: MvccSnapshot[applied={T|T < 7315367269181108224}]
I20260806 01:05:37.218348 1761 mvcc.cc:205] Tried to move back new op lower bound from 7315367269181108224 to 7315367269088608256. Current Snapshot: MvccSnapshot[applied={T|T < 7315367269181108224}]
I20260806 01:05:37.228381 1063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:37.228833 1063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:37.232168 1063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af9f89a085c64fd6908b74cde15c8606: Sending DeleteTablet for 3 replicas of tablet d05dd04fa5c246ac9b875f85d334c248
I20260806 01:05:37.233197 1251 tablet_service.cc:1514] Processing DeleteTablet for tablet d05dd04fa5c246ac9b875f85d334c248 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:37 UTC) from {username='slave'} at 127.0.0.1:54588
I20260806 01:05:37.233291 1443 tablet_service.cc:1514] Processing DeleteTablet for tablet d05dd04fa5c246ac9b875f85d334c248 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:37 UTC) from {username='slave'} at 127.0.0.1:47522
I20260806 01:05:37.233481 1630 tablet_service.cc:1514] Processing DeleteTablet for tablet d05dd04fa5c246ac9b875f85d334c248 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:37 UTC) from {username='slave'} at 127.0.0.1:41606
I20260806 01:05:37.238327 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 1163
I20260806 01:05:37.244095 1777 tablet_replica.cc:330] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: stopping tablet replica
I20260806 01:05:37.244684 1777 raft_consensus.cc:2229] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:37.245306 1777 raft_consensus.cc:2258] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:37.245889 1780 tablet_replica.cc:330] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: stopping tablet replica
I20260806 01:05:37.246662 1780 raft_consensus.cc:2229] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:37.248795 1778 tablet_replica.cc:330] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: stopping tablet replica
I20260806 01:05:37.249203 1778 raft_consensus.cc:2229] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:05:37.249462 1780 raft_consensus.cc:2258] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:37.249850 1777 ts_tablet_manager.cc:1905] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:37.249794 1778 raft_consensus.cc:2258] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:37.252188 1780 ts_tablet_manager.cc:1905] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:37.252280 1778 ts_tablet_manager.cc:1905] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:37.259413 1778 ts_tablet_manager.cc:1918] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:37.259634 1778 log.cc:1196] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/d05dd04fa5c246ac9b875f85d334c248
I20260806 01:05:37.260236 1778 ts_tablet_manager.cc:1939] T d05dd04fa5c246ac9b875f85d334c248 P 2bef9d2b8ae84f8d9ebb35fbf2a7192b: Deleting consensus metadata
I20260806 01:05:37.261049 1780 ts_tablet_manager.cc:1918] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:37.261304 1780 log.cc:1196] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/d05dd04fa5c246ac9b875f85d334c248
I20260806 01:05:37.261875 1780 ts_tablet_manager.cc:1939] T d05dd04fa5c246ac9b875f85d334c248 P 0fdb17178ff14fa98ef5397a3f3b9ebf: Deleting consensus metadata
I20260806 01:05:37.262049 1046 catalog_manager.cc:4841] TS 2bef9d2b8ae84f8d9ebb35fbf2a7192b (127.5.108.130:46275): tablet d05dd04fa5c246ac9b875f85d334c248 (table test-table [id=aa3a3b334b244b5c8f51b0670bc1609f]) successfully deleted
I20260806 01:05:37.263623 1044 catalog_manager.cc:4841] TS 0fdb17178ff14fa98ef5397a3f3b9ebf (127.5.108.131:42711): tablet d05dd04fa5c246ac9b875f85d334c248 (table test-table [id=aa3a3b334b244b5c8f51b0670bc1609f]) successfully deleted
I20260806 01:05:37.264091 1777 ts_tablet_manager.cc:1918] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:37.264258 1777 log.cc:1196] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/d05dd04fa5c246ac9b875f85d334c248
I20260806 01:05:37.264784 1777 ts_tablet_manager.cc:1939] T d05dd04fa5c246ac9b875f85d334c248 P f9afd1b7df3f402e9f4b36c51cdeffe8: Deleting consensus metadata
I20260806 01:05:37.266094 1042 catalog_manager.cc:4841] TS f9afd1b7df3f402e9f4b36c51cdeffe8 (127.5.108.129:44473): tablet d05dd04fa5c246ac9b875f85d334c248 (table test-table [id=aa3a3b334b244b5c8f51b0670bc1609f]) successfully deleted
I20260806 01:05:37.267668 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.239650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54596 (local address 127.5.108.129:44473)
0806 01:05:37.239853 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.239861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.253272 (+ 13411us) server_negotiation.cc:407] Connection header received
0806 01:05:37.253714 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.253719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.253828 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.253903 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:37.254545 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.255577 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.257400 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.258037 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.258129 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.266344 (+ 8215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:37.266358 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:37.266370 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:37.266863 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:37.266904 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:37.266928 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:37.267056 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.267191 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.267294 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:05:37.268415 1346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:37.354460 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 1163
I20260806 01:05:37.380657 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 1359
I20260806 01:05:37.394663 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.381725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.5.108.130:46275)
0806 01:05:37.381941 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.381947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.381973 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:05:37.382041 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.382046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.382175 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.382299 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:37.382668 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.383838 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.385245 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.385878 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.386029 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.393010 (+ 6981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:37.393021 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:37.393038 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:37.393810 (+ 772us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:37.393887 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:37.393920 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:37.394035 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.394163 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.394270 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:05:37.395459 1531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:37.502741 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 1359
I20260806 01:05:37.538506 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 1541
I20260806 01:05:37.548976 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:37.539438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.5.108.131:42711)
0806 01:05:37.541169 (+ 1731us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:37.541176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:37.541203 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:05:37.541271 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:37.541276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:37.541374 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:37.541495 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:37.542130 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.543217 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.545180 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:37.545788 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:37.545919 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:37.547088 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:37.547102 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:37.547128 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:37.548024 (+ 896us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:37.548087 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:37.548113 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:37.548247 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:37.548432 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 01:05:37.548570 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1589}
I20260806 01:05:37.549786 1708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:37.652810 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 1541
I20260806 01:05:37.674001 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 1017
I20260806 01:05:37.675176 1114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:37.779785 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 1017
2026-08-06T01:05:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2724 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:37 dist-test-slave-x4qh krb5kdc[1790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:37 dist-test-slave-x4qh krb5kdc[1790](info): set up 2 sockets
Aug 06 01:05:37 dist-test-slave-x4qh krb5kdc[1790](info): commencing operation
krb5kdc: starting...
W20260806 01:05:39.041962 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:39 dist-test-slave-x4qh krb5kdc[1790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:05:39.311990 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44065
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:35435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:44065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:39.639925 1809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:39.640174 1809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:39.640252 1809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:39.646620 1809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:39.646706 1809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:39.646754 1809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:39.646807 1809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:39.646842 1809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:39.655694 1809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:35435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:44065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:44065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:39.656785 1809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:39.657982 1809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:39.669497 1822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:39.669616 1819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:39.669652 1820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:39.670778 1809 server_base.cc:1034] running on GCE node
I20260806 01:05:39.671543 1809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:39.672850 1809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:39.674049 1809 hybrid_clock.cc:648] HybridClock initialized: now 1785978339674012 us; error 27 us; skew 500 ppm
I20260806 01:05:39.674386 1809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:39 dist-test-slave-x4qh krb5kdc[1790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978339, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:39.683876 1809 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:05:39.685353 1809 webserver.cc:460] Webserver started at http://127.5.108.190:37785/ using document root <none> and password file <none>
I20260806 01:05:39.685974 1809 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:39.686072 1809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:39.686318 1809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:39.689378 1809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "55d84e0e13f2468c8e28e03d2aa665f7"
format_stamp: "Formatted at 2026-08-06 01:05:39 on dist-test-slave-x4qh"
server_key: "586376e2d1009b0bc8f909d3eadb8a3a"
server_key_iv: "2b98b7c436f019a91cfd80a2219cadde"
server_key_version: "encryptionkey@0"
I20260806 01:05:39.690057 1809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "55d84e0e13f2468c8e28e03d2aa665f7"
format_stamp: "Formatted at 2026-08-06 01:05:39 on dist-test-slave-x4qh"
server_key: "586376e2d1009b0bc8f909d3eadb8a3a"
server_key_iv: "2b98b7c436f019a91cfd80a2219cadde"
server_key_version: "encryptionkey@0"
I20260806 01:05:39.694685 1809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:05:39.697845 1835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:39.699107 1809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:05:39.699285 1809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "55d84e0e13f2468c8e28e03d2aa665f7"
format_stamp: "Formatted at 2026-08-06 01:05:39 on dist-test-slave-x4qh"
server_key: "586376e2d1009b0bc8f909d3eadb8a3a"
server_key_iv: "2b98b7c436f019a91cfd80a2219cadde"
server_key_version: "encryptionkey@0"
I20260806 01:05:39.699522 1809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:39.716977 1809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:39.720692 1809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:39.720988 1809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:39.739220 1809 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:44065
I20260806 01:05:39.739253 1909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:44065 every 8 connection(s)
I20260806 01:05:39.740635 1809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:39.744294 1910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:39.748911 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 1809
I20260806 01:05:39.749118 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:39.749601 5554 external_mini_cluster.cc:1428] Setting key 72495cc8fb2ab121e2d323f9c0f1a010
I20260806 01:05:39.756040 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7: Bootstrap starting.
I20260806 01:05:39.759718 1910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:39.760835 1910 log.cc:824] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:39.763733 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7: No bootstrap required, opened a new log
Aug 06 01:05:39 dist-test-slave-x4qh krb5kdc[1790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:39.772830 1910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d84e0e13f2468c8e28e03d2aa665f7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44065 } }
I20260806 01:05:39.773196 1910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:39.773319 1910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d84e0e13f2468c8e28e03d2aa665f7, State: Initialized, Role: FOLLOWER
I20260806 01:05:39.773841 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:39.751309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.5.108.190:44065)
0806 01:05:39.752154 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:39.752168 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:39.752223 (+ 55us) server_negotiation.cc:407] Connection header received
0806 01:05:39.754640 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:39.754680 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:39.755146 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:39.755562 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:39.756859 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:39.758920 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:39.760288 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:39.761359 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:39.761549 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:39.765903 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:39.765913 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:39.765936 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:39.766004 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:39.769647 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:39.770507 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:39.770513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:39.770518 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:39.770681 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:39.771115 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:39.771118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:39.771120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:39.771688 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:39.771917 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:39.772353 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 01:05:39.772631 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":336,"threads_started":1}
I20260806 01:05:39.773910 1910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d84e0e13f2468c8e28e03d2aa665f7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44065 } }
I20260806 01:05:39.774237 1910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:39.774389 1910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:39.774572 1910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:39.776854 1910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d84e0e13f2468c8e28e03d2aa665f7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44065 } }
I20260806 01:05:39.777316 1910 leader_election.cc:302] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55d84e0e13f2468c8e28e03d2aa665f7; no voters:
I20260806 01:05:39.777848 1910 leader_election.cc:288] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:39.777984 1919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:39.778680 1919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [term 1 LEADER]: Becoming Leader. State: Replica: 55d84e0e13f2468c8e28e03d2aa665f7, State: Running, Role: LEADER
I20260806 01:05:39.779143 1919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d84e0e13f2468c8e28e03d2aa665f7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44065 } }
I20260806 01:05:39.779883 1910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:39.781951 1922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55d84e0e13f2468c8e28e03d2aa665f7. Latest consensus state: current_term: 1 leader_uuid: "55d84e0e13f2468c8e28e03d2aa665f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d84e0e13f2468c8e28e03d2aa665f7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44065 } } }
I20260806 01:05:39.782305 1922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:39.783272 1921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55d84e0e13f2468c8e28e03d2aa665f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d84e0e13f2468c8e28e03d2aa665f7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 44065 } } }
I20260806 01:05:39.783728 1921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:39.784495 1931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:39.788954 1931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:39.797607 1931 catalog_manager.cc:1347] Generated new cluster ID: c66b678524544d5db58cdd8d669f22a9
I20260806 01:05:39.797765 1931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:39.811589 1931 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:39.812778 1931 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:39.821707 1931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7: Generated new TSK 0
I20260806 01:05:39.822436 1931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.108.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
Entry for principal kudu/127.5.108.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab.
I20260806 01:05:39.891640 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44065
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:35435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:40.228819 1950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:40.229055 1950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:40.229146 1950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:40.235327 1950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:40.235476 1950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:40.235599 1950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:05:40.244824 1950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:35435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.129.keytab
--principal=kudu/127.5.108.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:40.245851 1950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:40.246999 1950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:40.259269 1961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:40.259431 1959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:40.259621 1957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:40.260421 1950 server_base.cc:1034] running on GCE node
I20260806 01:05:40.261034 1950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:40.261866 1950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:40.263052 1950 hybrid_clock.cc:648] HybridClock initialized: now 1785978340262999 us; error 51 us; skew 500 ppm
I20260806 01:05:40.263357 1950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:40 dist-test-slave-x4qh krb5kdc[1790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:40.266431 1950 init.cc:377] Logged in from keytab as kudu/127.5.108.129@KRBTEST.COM (short username kudu)
I20260806 01:05:40.267777 1950 webserver.cc:460] Webserver started at http://127.5.108.129:34761/ using document root <none> and password file <none>
I20260806 01:05:40.268411 1950 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:40.268559 1950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:40.268879 1950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:40.271533 1950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "f90602635dca478ba2cf8192433df86d"
format_stamp: "Formatted at 2026-08-06 01:05:40 on dist-test-slave-x4qh"
server_key: "fd870e600001fadb9f80b0600801666d"
server_key_iv: "829136eac214d8f8b5cf2695253ce3e9"
server_key_version: "encryptionkey@0"
I20260806 01:05:40.272291 1950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "f90602635dca478ba2cf8192433df86d"
format_stamp: "Formatted at 2026-08-06 01:05:40 on dist-test-slave-x4qh"
server_key: "fd870e600001fadb9f80b0600801666d"
server_key_iv: "829136eac214d8f8b5cf2695253ce3e9"
server_key_version: "encryptionkey@0"
I20260806 01:05:40.276808 1950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:05:40.280001 1969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:40.281260 1950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:40.281418 1950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "f90602635dca478ba2cf8192433df86d"
format_stamp: "Formatted at 2026-08-06 01:05:40 on dist-test-slave-x4qh"
server_key: "fd870e600001fadb9f80b0600801666d"
server_key_iv: "829136eac214d8f8b5cf2695253ce3e9"
server_key_version: "encryptionkey@0"
I20260806 01:05:40.281594 1950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:40.293711 1950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:40.297288 1950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:40.297571 1950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:40.298488 1950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:40.299821 1950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:40.299917 1950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:40.300076 1950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:40.300161 1950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:40.331838 1950 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:46745
I20260806 01:05:40.331887 2127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:46745 every 8 connection(s)
I20260806 01:05:40.333076 1950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:05:40.333374 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 1950
I20260806 01:05:40.333546 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:05:40.334095 5554 external_mini_cluster.cc:1428] Setting key d7ad244a2a2bd0f1b5aa9a4a222b4c47
Aug 06 01:05:40 dist-test-slave-x4qh krb5kdc[1790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.129@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:40.354140 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:40.335839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:41613 (local address 127.5.108.190:44065)
0806 01:05:40.336371 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:40.336378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:40.337467 (+ 1089us) server_negotiation.cc:407] Connection header received
0806 01:05:40.339242 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:40.339250 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:40.339343 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:40.339500 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:40.340859 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:40.341974 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:40.343743 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:40.344404 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:40.344551 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:40.347911 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:40.347919 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:40.347926 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:40.347968 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:40.351056 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:40.351992 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:40.351995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:40.351998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:40.352110 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:40.352551 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:40.352556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:40.352559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:40.352831 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:40.352979 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:40.353598 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 01:05:40.353815 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":255,"threads_started":1}
I20260806 01:05:40.354959 2131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:40.336441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44065 (local address 127.5.108.129:41613)
0806 01:05:40.337292 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:40.337345 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:40.338988 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:40.339709 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:40.339716 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:40.340162 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:40.340677 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:40.340694 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:40.342149 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:40.342153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:40.343315 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:40.343326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:40.344608 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:40.344612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:40.344809 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:40.345468 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:40.345489 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:40.347712 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:40.351202 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:40.351212 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:40.351227 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:40.351883 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:40.352246 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:40.352249 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:40.352252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:40.352427 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:40.352953 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:40.352960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:40.353260 (+ 300us) client_negotiation.cc:769] Sending connection context
0806 01:05:40.353493 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 01:05:40.353947 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":401,"threads_started":1}
I20260806 01:05:40.356443 2128 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44065
I20260806 01:05:40.356737 2128 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:40.357388 2128 heartbeater.cc:511] Master 127.5.108.190:44065 requested a full tablet report, sending...
I20260806 01:05:40.359688 1855 ts_manager.cc:194] Registered new tserver with Master: f90602635dca478ba2cf8192433df86d (127.5.108.129:46745)
I20260806 01:05:40.361832 1855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.129@KRBTEST.COM'} at 127.5.108.129:41613
WARNING: no policy specified for kudu/127.5.108.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
Entry for principal kudu/127.5.108.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab.
I20260806 01:05:40.402902 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44065
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:35435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:40.721067 2135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:40.721306 2135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:40.721393 2135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:40.727928 2135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:40.728031 2135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:40.728132 2135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:05:40.736629 2135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:35435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.130.keytab
--principal=kudu/127.5.108.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:40.737741 2135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:40.738997 2135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:40.751713 2142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:40.751724 2144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:40.753337 2135 server_base.cc:1034] running on GCE node
W20260806 01:05:40.753715 2148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:40.754403 2135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:40.755462 2135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:40.756698 2135 hybrid_clock.cc:648] HybridClock initialized: now 1785978340756656 us; error 33 us; skew 500 ppm
I20260806 01:05:40.757107 2135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:40 dist-test-slave-x4qh krb5kdc[1790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:40.760128 2135 init.cc:377] Logged in from keytab as kudu/127.5.108.130@KRBTEST.COM (short username kudu)
I20260806 01:05:40.761322 2135 webserver.cc:460] Webserver started at http://127.5.108.130:45197/ using document root <none> and password file <none>
I20260806 01:05:40.761921 2135 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:40.762017 2135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:40.762274 2135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:40.764884 2135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "75379e425f594206a30e2c5997f23fe5"
format_stamp: "Formatted at 2026-08-06 01:05:40 on dist-test-slave-x4qh"
server_key: "4c0355d9a9b0d51251f9522c71b4858f"
server_key_iv: "ce1691960af42171fb251c39e2151c40"
server_key_version: "encryptionkey@0"
I20260806 01:05:40.765592 2135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "75379e425f594206a30e2c5997f23fe5"
format_stamp: "Formatted at 2026-08-06 01:05:40 on dist-test-slave-x4qh"
server_key: "4c0355d9a9b0d51251f9522c71b4858f"
server_key_iv: "ce1691960af42171fb251c39e2151c40"
server_key_version: "encryptionkey@0"
I20260806 01:05:40.770102 2135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:05:40.773313 2155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:40.774415 2135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:05:40.774597 2135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "75379e425f594206a30e2c5997f23fe5"
format_stamp: "Formatted at 2026-08-06 01:05:40 on dist-test-slave-x4qh"
server_key: "4c0355d9a9b0d51251f9522c71b4858f"
server_key_iv: "ce1691960af42171fb251c39e2151c40"
server_key_version: "encryptionkey@0"
I20260806 01:05:40.774785 2135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:40.796656 2135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:40.800491 2135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:40.800762 2135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:40.801663 2135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:40.802879 2135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:40.802987 2135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:40.803110 2135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:40.803180 2135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:40.836408 2135 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:44017
I20260806 01:05:40.836493 2319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:44017 every 8 connection(s)
I20260806 01:05:40.837634 2135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:05:40.838799 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 2135
I20260806 01:05:40.838994 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:05:40.839612 5554 external_mini_cluster.cc:1428] Setting key 66297ff3839aff387bd378065b9eafa5
Aug 06 01:05:40 dist-test-slave-x4qh krb5kdc[1790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.130@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:40.860345 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:40.840368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:41039 (local address 127.5.108.190:44065)
0806 01:05:40.840577 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:40.840583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:40.841861 (+ 1278us) server_negotiation.cc:407] Connection header received
0806 01:05:40.844665 (+ 2804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:40.844669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:40.844740 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:40.844820 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:40.846801 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:40.847530 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:40.850055 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:40.850455 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:40.850551 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:40.854683 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:40.854688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:40.854691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:40.854728 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:40.857312 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:40.857994 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:40.857999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:40.858003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:40.858153 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:40.858615 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:40.858621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:40.858625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:40.858883 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:40.859059 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:40.859781 (+ 722us) server_negotiation.cc:299] Negotiation successful
0806 01:05:40.860010 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:05:40.861255 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:40.840878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44065 (local address 127.5.108.130:41039)
0806 01:05:40.841679 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:40.841756 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:40.844351 (+ 2595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:40.845087 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:40.845099 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:40.845863 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:40.846634 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:40.846660 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:40.847680 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:40.847687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:40.849813 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:40.849830 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:40.850676 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:40.850685 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:40.851049 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:40.851790 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:40.851823 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:40.854464 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:40.857453 (+ 2989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:40.857461 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:40.857475 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:40.857873 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:40.858300 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:40.858305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:40.858309 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:40.858498 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:40.859044 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:40.859051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:40.859358 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 01:05:40.859688 (+ 330us) client_negotiation.cc:240] Negotiation successful
0806 01:05:40.860172 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":328,"threads_started":1}
I20260806 01:05:40.862633 2320 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44065
I20260806 01:05:40.862938 2320 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:40.863870 2320 heartbeater.cc:511] Master 127.5.108.190:44065 requested a full tablet report, sending...
I20260806 01:05:40.865527 1855 ts_manager.cc:194] Registered new tserver with Master: 75379e425f594206a30e2c5997f23fe5 (127.5.108.130:44017)
I20260806 01:05:40.866816 1855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.130@KRBTEST.COM'} at 127.5.108.130:41039
WARNING: no policy specified for kudu/127.5.108.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
Entry for principal kudu/127.5.108.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab.
I20260806 01:05:40.909632 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44065
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:35435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:41.233844 2325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:41.234078 2325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:41.234161 2325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:41.240926 2325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:41.241031 2325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:41.241132 2325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:05:41.249794 2325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:35435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.131.keytab
--principal=kudu/127.5.108.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:44065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:41.250756 2325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:41.251868 2325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:41.264024 2338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:41.263990 2340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:41.263990 2335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:41.264779 2325 server_base.cc:1034] running on GCE node
I20260806 01:05:41.265372 2325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:41.266310 2325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:41.267468 2325 hybrid_clock.cc:648] HybridClock initialized: now 1785978341267420 us; error 52 us; skew 500 ppm
I20260806 01:05:41.267760 2325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:41 dist-test-slave-x4qh krb5kdc[1790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978341, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:41.270646 2325 init.cc:377] Logged in from keytab as kudu/127.5.108.131@KRBTEST.COM (short username kudu)
I20260806 01:05:41.272081 2325 webserver.cc:460] Webserver started at http://127.5.108.131:41743/ using document root <none> and password file <none>
I20260806 01:05:41.272723 2325 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:41.272873 2325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:41.273169 2325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:41.275846 2325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "804636919b6246059a57f3350727cf47"
format_stamp: "Formatted at 2026-08-06 01:05:41 on dist-test-slave-x4qh"
server_key: "052acfbccb639182a1f9763fe30dd9ec"
server_key_iv: "77833b15a3483f76bf4294ba3c8fde50"
server_key_version: "encryptionkey@0"
I20260806 01:05:41.276656 2325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "804636919b6246059a57f3350727cf47"
format_stamp: "Formatted at 2026-08-06 01:05:41 on dist-test-slave-x4qh"
server_key: "052acfbccb639182a1f9763fe30dd9ec"
server_key_iv: "77833b15a3483f76bf4294ba3c8fde50"
server_key_version: "encryptionkey@0"
I20260806 01:05:41.281651 2325 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 01:05:41.284924 2350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:41.286098 2325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:05:41.286274 2325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "804636919b6246059a57f3350727cf47"
format_stamp: "Formatted at 2026-08-06 01:05:41 on dist-test-slave-x4qh"
server_key: "052acfbccb639182a1f9763fe30dd9ec"
server_key_iv: "77833b15a3483f76bf4294ba3c8fde50"
server_key_version: "encryptionkey@0"
I20260806 01:05:41.286449 2325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:41.307843 2325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:41.311477 2325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:41.311764 2325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:41.312664 2325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:41.313868 2325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:41.313967 2325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:41.314114 2325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:41.314183 2325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:41.347059 2325 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:43565
I20260806 01:05:41.347143 2513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:43565 every 8 connection(s)
I20260806 01:05:41.348316 2325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:05:41.357547 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 2325
I20260806 01:05:41.357822 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:05:41.358361 5554 external_mini_cluster.cc:1428] Setting key 2f00e596e149bba88bd35c15c927f3c6
Aug 06 01:05:41 dist-test-slave-x4qh krb5kdc[1790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978341, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.131@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:05:41.364640 2128 heartbeater.cc:503] Master 127.5.108.190:44065 was elected leader, sending a full tablet report...
I20260806 01:05:41.367515 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.350840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:55487 (local address 127.5.108.190:44065)
0806 01:05:41.351041 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.351046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.352034 (+ 988us) server_negotiation.cc:407] Connection header received
0806 01:05:41.353901 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.353907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.354022 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.354148 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:41.355626 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.356394 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.357988 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.358544 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.358680 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.362129 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:41.362133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:41.362136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:41.362164 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:41.364468 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:41.365251 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:41.365256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:41.365259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:41.365416 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:41.365922 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:41.365925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:41.365927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:41.366237 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:41.366382 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.366980 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.367177 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:05:41.368396 2517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.351157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:44065 (local address 127.5.108.131:55487)
0806 01:05:41.351876 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:41.351926 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:41.353638 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:41.354388 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:41.354395 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:41.354862 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:41.355439 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.355472 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.356540 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.356545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.357825 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.357837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.358768 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.358777 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.359061 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.359889 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:05:41.359910 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:41.361941 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:41.364601 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:41.364611 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:41.364624 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:41.365133 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:41.365531 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:05:41.365535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:05:41.365538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:05:41.365808 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:05:41.366365 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:41.366372 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:41.366646 (+ 274us) client_negotiation.cc:769] Sending connection context
0806 01:05:41.366875 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 01:05:41.367316 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":312,"threads_started":1}
I20260806 01:05:41.369626 2514 heartbeater.cc:347] Connected to a master server at 127.5.108.190:44065
I20260806 01:05:41.369901 2514 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:41.370536 2514 heartbeater.cc:511] Master 127.5.108.190:44065 requested a full tablet report, sending...
I20260806 01:05:41.371933 1855 ts_manager.cc:194] Registered new tserver with Master: 804636919b6246059a57f3350727cf47 (127.5.108.131:43565)
I20260806 01:05:41.372884 1855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.108.131@KRBTEST.COM'} at 127.5.108.131:55487
I20260806 01:05:41.373462 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:05:41.388824 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.377201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.5.108.190:44065)
0806 01:05:41.377337 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.377341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.377805 (+ 464us) server_negotiation.cc:407] Connection header received
0806 01:05:41.377917 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.377920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.377970 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.378041 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:41.379439 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.380141 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.381380 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.381770 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.381897 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.384027 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:41.384031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:41.384033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:41.384058 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:41.386189 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:41.386826 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:41.386831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:41.386834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:41.386952 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:41.387299 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:41.387307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:41.387310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:41.387694 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:41.387958 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.388309 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.388517 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 01:05:41.392418 1855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:05:41.398797 1855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:05:41.421449 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.411120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:44017 (local address 127.0.0.1:35586)
0806 01:05:41.412721 (+ 1601us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:41.412739 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:41.412850 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:41.414181 (+ 1331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:41.414185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:41.414210 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:41.414602 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.414613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.416528 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.416533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.418396 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.418409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.420316 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.420322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.420578 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.420634 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 01:05:41.420756 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 01:05:41.420931 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1472,"mutex_wait_us":50,"thread_start_us":1340,"threads_started":1}
I20260806 01:05:41.421623 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.410978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45490 (local address 127.5.108.129:46745)
0806 01:05:41.412842 (+ 1864us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.412852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.412880 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:05:41.412976 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.412984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.413238 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.413454 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:41.414386 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.415971 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.418738 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.420057 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.420214 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.420456 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.421101 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.421286 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1686,"thread_start_us":394,"threads_started":1}
I20260806 01:05:41.421509 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.410686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:46745 (local address 127.0.0.1:45490)
0806 01:05:41.412112 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:41.412180 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:41.412335 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:41.413732 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:41.413738 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:41.413769 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:41.414181 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.414192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.416105 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.416110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.418568 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.418583 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.420596 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.420604 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.420754 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.420783 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 01:05:41.420857 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 01:05:41.420951 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1201,"mutex_wait_us":23,"thread_start_us":1063,"threads_started":1}
I20260806 01:05:41.421545 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.413120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:43565 (local address 127.0.0.1:52932)
0806 01:05:41.413928 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:41.413948 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:41.414064 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:41.414955 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:41.414959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:41.414981 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:41.415310 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.415318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.416846 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.416852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.418920 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.418930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.420271 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.420280 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.420881 (+ 601us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.420914 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:05:41.421004 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 01:05:41.421169 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":350,"thread_start_us":860,"threads_started":1}
I20260806 01:05:41.422184 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.413713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52932 (local address 127.5.108.131:43565)
0806 01:05:41.414576 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.414581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.414604 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:05:41.414668 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.414673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.414796 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.414963 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:41.415475 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.416686 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.419065 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.420015 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.420475 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.420709 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.421683 (+ 974us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.421843 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":645,"threads_started":1}
I20260806 01:05:41.423018 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.411714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.5.108.130:44017)
0806 01:05:41.413621 (+ 1907us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.413629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.413653 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:05:41.413747 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.413753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.413949 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.414805 (+ 856us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:41.415024 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.416357 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.418554 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.420042 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.422318 (+ 2276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.422482 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.422546 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.422693 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1754,"thread_start_us":1641,"threads_started":1}
I20260806 01:05:41.426115 2229 tablet_service.cc:1467] Processing CreateTablet for tablet 124e58faaa2849668e7607f919f9bedc (DEFAULT_TABLE table=test-table [id=470b98daef564e5ab506c6dc75777462]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:41.425985 2037 tablet_service.cc:1467] Processing CreateTablet for tablet 124e58faaa2849668e7607f919f9bedc (DEFAULT_TABLE table=test-table [id=470b98daef564e5ab506c6dc75777462]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:41.426259 2414 tablet_service.cc:1467] Processing CreateTablet for tablet 124e58faaa2849668e7607f919f9bedc (DEFAULT_TABLE table=test-table [id=470b98daef564e5ab506c6dc75777462]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:41.427286 2229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 124e58faaa2849668e7607f919f9bedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:41.427418 2037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 124e58faaa2849668e7607f919f9bedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:41.427423 2414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 124e58faaa2849668e7607f919f9bedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:41.437494 2542 tablet_bootstrap.cc:492] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: Bootstrap starting.
I20260806 01:05:41.440585 2543 tablet_bootstrap.cc:492] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: Bootstrap starting.
I20260806 01:05:41.440807 2542 tablet_bootstrap.cc:654] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:41.441341 2546 tablet_bootstrap.cc:492] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: Bootstrap starting.
I20260806 01:05:41.442454 2542 log.cc:824] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:41.444479 2543 tablet_bootstrap.cc:654] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:41.444921 2546 tablet_bootstrap.cc:654] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:41.445465 2543 log.cc:824] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:41.445735 2542 tablet_bootstrap.cc:492] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: No bootstrap required, opened a new log
I20260806 01:05:41.445988 2542 ts_tablet_manager.cc:1397] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260806 01:05:41.446125 2546 log.cc:824] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:41.447938 2543 tablet_bootstrap.cc:492] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: No bootstrap required, opened a new log
I20260806 01:05:41.448180 2543 ts_tablet_manager.cc:1397] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 01:05:41.453640 2546 tablet_bootstrap.cc:492] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: No bootstrap required, opened a new log
I20260806 01:05:41.453871 2546 ts_tablet_manager.cc:1397] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260806 01:05:41.455211 2543 raft_consensus.cc:348] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } }
I20260806 01:05:41.455608 2543 raft_consensus.cc:374] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:41.455732 2543 raft_consensus.cc:729] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 804636919b6246059a57f3350727cf47, State: Initialized, Role: FOLLOWER
I20260806 01:05:41.456344 2543 consensus_queue.cc:260] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } }
I20260806 01:05:41.457049 2542 raft_consensus.cc:348] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } }
I20260806 01:05:41.457383 2514 heartbeater.cc:503] Master 127.5.108.190:44065 was elected leader, sending a full tablet report...
I20260806 01:05:41.457558 2542 raft_consensus.cc:374] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:41.457721 2542 raft_consensus.cc:729] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75379e425f594206a30e2c5997f23fe5, State: Initialized, Role: FOLLOWER
I20260806 01:05:41.458457 2542 consensus_queue.cc:260] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } }
I20260806 01:05:41.458828 2543 ts_tablet_manager.cc:1428] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260806 01:05:41.462394 2320 heartbeater.cc:503] Master 127.5.108.190:44065 was elected leader, sending a full tablet report...
I20260806 01:05:41.464170 2542 ts_tablet_manager.cc:1428] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: Time spent starting tablet: real 0.018s user 0.014s sys 0.002s
I20260806 01:05:41.464974 2546 raft_consensus.cc:348] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } }
I20260806 01:05:41.465474 2546 raft_consensus.cc:374] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:41.465595 2546 raft_consensus.cc:729] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f90602635dca478ba2cf8192433df86d, State: Initialized, Role: FOLLOWER
I20260806 01:05:41.466092 2546 consensus_queue.cc:260] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } }
I20260806 01:05:41.467983 2546 ts_tablet_manager.cc:1428] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260806 01:05:41.585547 2129 tablet.cc:2369] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:05:41.589871 2321 tablet.cc:2369] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:05:41.599866 2515 tablet.cc:2369] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:05:41.658021 2558 raft_consensus.cc:482] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:41.658378 2558 raft_consensus.cc:504] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } }
I20260806 01:05:41.660264 2558 leader_election.cc:288] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f90602635dca478ba2cf8192433df86d (127.5.108.129:46745), 804636919b6246059a57f3350727cf47 (127.5.108.131:43565)
I20260806 01:05:41.666755 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.660768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:44329 (local address 127.5.108.131:43565)
0806 01:05:41.660955 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.660961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.661575 (+ 614us) server_negotiation.cc:407] Connection header received
0806 01:05:41.661730 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.661741 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.661846 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.661940 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:41.662667 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.663370 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.665122 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.665851 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.665954 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.666031 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.666387 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.666506 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 01:05:41.666786 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.660553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:38505 (local address 127.5.108.129:46745)
0806 01:05:41.660800 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.660809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.661446 (+ 637us) server_negotiation.cc:407] Connection header received
0806 01:05:41.661608 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.661614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.661721 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.661814 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:41.662534 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.663281 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.665168 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.665928 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.666070 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.666135 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.666462 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.666581 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:05:41.666882 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.660497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:43565 (local address 127.5.108.130:44329)
0806 01:05:41.661448 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:41.661466 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:41.661607 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:41.661985 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:41.661988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:41.662019 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:41.662426 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.662436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.663505 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.663509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.664970 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.664982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.666052 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.666058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.666246 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.666268 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 01:05:41.666333 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 01:05:41.666477 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":61,"thread_start_us":466,"threads_started":1}
I20260806 01:05:41.666906 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.660495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:46745 (local address 127.5.108.130:38505)
0806 01:05:41.661341 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:41.661359 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:41.661473 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:41.661859 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:41.661863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:41.661893 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:41.662382 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.662392 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.663426 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.663433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.664985 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:41.664997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.666132 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.666135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:41.666284 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.666305 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 01:05:41.666407 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 01:05:41.666518 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":99,"spinlock_wait_cycles":48512,"thread_start_us":527,"threads_started":1}
I20260806 01:05:41.667774 2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "124e58faaa2849668e7607f919f9bedc" candidate_uuid: "75379e425f594206a30e2c5997f23fe5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f90602635dca478ba2cf8192433df86d" is_pre_election: true
I20260806 01:05:41.667821 2445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "124e58faaa2849668e7607f919f9bedc" candidate_uuid: "75379e425f594206a30e2c5997f23fe5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804636919b6246059a57f3350727cf47" is_pre_election: true
I20260806 01:05:41.668239 2073 raft_consensus.cc:2454] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75379e425f594206a30e2c5997f23fe5 in term 0.
I20260806 01:05:41.668246 2445 raft_consensus.cc:2454] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75379e425f594206a30e2c5997f23fe5 in term 0.
I20260806 01:05:41.668905 2160 leader_election.cc:302] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75379e425f594206a30e2c5997f23fe5, f90602635dca478ba2cf8192433df86d; no voters:
I20260806 01:05:41.669305 2558 raft_consensus.cc:2781] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:05:41.669447 2558 raft_consensus.cc:482] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:05:41.669600 2558 raft_consensus.cc:3037] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:41.671903 2558 raft_consensus.cc:504] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } }
I20260806 01:05:41.672524 2558 leader_election.cc:288] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [CANDIDATE]: Term 1 election: Requested vote from peers f90602635dca478ba2cf8192433df86d (127.5.108.129:46745), 804636919b6246059a57f3350727cf47 (127.5.108.131:43565)
I20260806 01:05:41.672927 2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "124e58faaa2849668e7607f919f9bedc" candidate_uuid: "75379e425f594206a30e2c5997f23fe5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f90602635dca478ba2cf8192433df86d"
I20260806 01:05:41.673026 2445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "124e58faaa2849668e7607f919f9bedc" candidate_uuid: "75379e425f594206a30e2c5997f23fe5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804636919b6246059a57f3350727cf47"
I20260806 01:05:41.673125 2073 raft_consensus.cc:3037] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:41.673228 2445 raft_consensus.cc:3037] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:41.675280 2445 raft_consensus.cc:2454] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75379e425f594206a30e2c5997f23fe5 in term 1.
I20260806 01:05:41.675537 2073 raft_consensus.cc:2454] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75379e425f594206a30e2c5997f23fe5 in term 1.
I20260806 01:05:41.675747 2158 leader_election.cc:302] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75379e425f594206a30e2c5997f23fe5, 804636919b6246059a57f3350727cf47; no voters:
I20260806 01:05:41.676119 2558 raft_consensus.cc:2781] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:41.676610 2558 raft_consensus.cc:686] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 1 LEADER]: Becoming Leader. State: Replica: 75379e425f594206a30e2c5997f23fe5, State: Running, Role: LEADER
I20260806 01:05:41.677066 2558 consensus_queue.cc:237] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } }
I20260806 01:05:41.681785 1855 catalog_manager.cc:5495] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75379e425f594206a30e2c5997f23fe5 (127.5.108.130). New cstate: current_term: 1 leader_uuid: "75379e425f594206a30e2c5997f23fe5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75379e425f594206a30e2c5997f23fe5" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 44017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 } health_report { overall_health: UNKNOWN } } }
I20260806 01:05:41.714583 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.709894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.5.108.130:44017)
0806 01:05:41.710054 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.710059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.710072 (+ 13us) server_negotiation.cc:407] Connection header received
0806 01:05:41.710140 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.710143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.710222 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.710329 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:05:41.710614 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.711294 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.712821 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.713202 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.713305 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.713676 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:05:41.714140 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:05:41.714243 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.714329 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.714388 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 01:05:41.722225 2073 raft_consensus.cc:1261] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [term 1 FOLLOWER]: Refusing update from remote peer 75379e425f594206a30e2c5997f23fe5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:41.722225 2445 raft_consensus.cc:1261] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [term 1 FOLLOWER]: Refusing update from remote peer 75379e425f594206a30e2c5997f23fe5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:41.723234 2558 consensus_queue.cc:1035] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "804636919b6246059a57f3350727cf47" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:41.723523 2565 consensus_queue.cc:1035] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f90602635dca478ba2cf8192433df86d" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 46745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:41.748553 1855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:41.748867 1855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:41.752921 1855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55d84e0e13f2468c8e28e03d2aa665f7: Sending DeleteTablet for 3 replicas of tablet 124e58faaa2849668e7607f919f9bedc
I20260806 01:05:41.753942 2229 tablet_service.cc:1514] Processing DeleteTablet for tablet 124e58faaa2849668e7607f919f9bedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:41 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:35586
I20260806 01:05:41.754642 2037 tablet_service.cc:1514] Processing DeleteTablet for tablet 124e58faaa2849668e7607f919f9bedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:41 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:45490
I20260806 01:05:41.755543 2584 tablet_replica.cc:330] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: stopping tablet replica
I20260806 01:05:41.755991 2584 raft_consensus.cc:2229] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:41.756501 2584 raft_consensus.cc:2258] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:41.757766 2414 tablet_service.cc:1514] Processing DeleteTablet for tablet 124e58faaa2849668e7607f919f9bedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:41 UTC) from {username='kudu', principal='kudu/127.5.108.190@KRBTEST.COM'} at 127.0.0.1:52932
I20260806 01:05:41.758085 2583 tablet_replica.cc:330] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: stopping tablet replica
I20260806 01:05:41.758577 2584 ts_tablet_manager.cc:1905] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:41.758535 2583 raft_consensus.cc:2229] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:05:41.759410 2583 raft_consensus.cc:2258] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:41.762089 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 1950
I20260806 01:05:41.765506 2584 ts_tablet_manager.cc:1918] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:41.765698 2584 log.cc:1196] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/124e58faaa2849668e7607f919f9bedc
I20260806 01:05:41.766263 2584 ts_tablet_manager.cc:1939] T 124e58faaa2849668e7607f919f9bedc P f90602635dca478ba2cf8192433df86d: Deleting consensus metadata
I20260806 01:05:41.767956 1838 catalog_manager.cc:4841] TS f90602635dca478ba2cf8192433df86d (127.5.108.129:46745): tablet 124e58faaa2849668e7607f919f9bedc (table test-table [id=470b98daef564e5ab506c6dc75777462]) successfully deleted
I20260806 01:05:41.768546 2585 tablet_replica.cc:330] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: stopping tablet replica
I20260806 01:05:41.769024 2585 raft_consensus.cc:2229] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:41.769615 2585 raft_consensus.cc:2258] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:41.771631 2583 ts_tablet_manager.cc:1905] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:41.773448 2585 ts_tablet_manager.cc:1905] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:41.780900 2583 ts_tablet_manager.cc:1918] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:41.781098 2585 ts_tablet_manager.cc:1918] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:41.781163 2583 log.cc:1196] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/124e58faaa2849668e7607f919f9bedc
I20260806 01:05:41.781327 2585 log.cc:1196] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/124e58faaa2849668e7607f919f9bedc
I20260806 01:05:41.781819 2583 ts_tablet_manager.cc:1939] T 124e58faaa2849668e7607f919f9bedc P 75379e425f594206a30e2c5997f23fe5: Deleting consensus metadata
I20260806 01:05:41.781899 2585 ts_tablet_manager.cc:1939] T 124e58faaa2849668e7607f919f9bedc P 804636919b6246059a57f3350727cf47: Deleting consensus metadata
I20260806 01:05:41.783555 1836 catalog_manager.cc:4841] TS 75379e425f594206a30e2c5997f23fe5 (127.5.108.130:44017): tablet 124e58faaa2849668e7607f919f9bedc (table test-table [id=470b98daef564e5ab506c6dc75777462]) successfully deleted
I20260806 01:05:41.783955 1836 catalog_manager.cc:4841] TS 804636919b6246059a57f3350727cf47 (127.5.108.131:43565): tablet 124e58faaa2849668e7607f919f9bedc (table test-table [id=470b98daef564e5ab506c6dc75777462]) successfully deleted
Aug 06 01:05:41 dist-test-slave-x4qh krb5kdc[1790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.129@KRBTEST.COM
I20260806 01:05:41.795094 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.764578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45502 (local address 127.5.108.129:46745)
0806 01:05:41.764806 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.764815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.778724 (+ 13909us) server_negotiation.cc:407] Connection header received
0806 01:05:41.778905 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.778912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.779004 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.779111 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:41.781068 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.782135 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.783883 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.784496 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.784607 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.789397 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:41.789410 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:41.789467 (+ 57us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:41.789519 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:41.792449 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:41.793270 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:41.793277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:41.793279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:41.793403 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:41.793788 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:41.793791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:41.793793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:41.794109 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:41.794310 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.794646 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.794801 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:05:41.795835 2122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:41.895458 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 1950
I20260806 01:05:41.931357 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 2135
Aug 06 01:05:41 dist-test-slave-x4qh krb5kdc[1790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.130@KRBTEST.COM
I20260806 01:05:41.948961 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:41.932288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.5.108.130:44017)
0806 01:05:41.932496 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:41.932501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:41.932522 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:05:41.932623 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:41.932628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:41.932707 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:41.932835 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:41.934049 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.935206 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.936627 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:41.937300 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:41.937433 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:41.941826 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:41.941845 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:41.941871 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:41.941921 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:41.945930 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:41.946594 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:41.946611 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:41.946614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:41.946771 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:41.947118 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:41.947123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:41.947127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:41.947660 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:41.948042 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:41.948309 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 01:05:41.948524 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:05:41.949776 2313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:42.062314 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 2135
I20260806 01:05:42.091741 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 2325
Aug 06 01:05:42 dist-test-slave-x4qh krb5kdc[1790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.131@KRBTEST.COM
I20260806 01:05:42.106469 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:42.092497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.5.108.131:43565)
0806 01:05:42.092678 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:42.092682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:42.092700 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:05:42.092867 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:42.092871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:42.092932 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:42.093014 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:42.094376 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:42.095108 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:42.096345 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:42.096710 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:42.096797 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:42.101113 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:42.101126 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:42.101140 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:42.101169 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:42.104000 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:42.104675 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:42.104683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:42.104686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:42.104811 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:42.105116 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:42.105119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:42.105121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:42.105470 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:42.105695 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:42.105927 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 01:05:42.106094 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:05:42.107059 2508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:42.191679 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 2325
I20260806 01:05:42.217276 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 1809
I20260806 01:05:42.218319 1903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:42.321472 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 1809
2026-08-06T01:05:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4540 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T01:05:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:42Z Disabled control of system clock
I20260806 01:05:42.413998 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:45031
--webserver_interface=127.5.108.190
--webserver_port=0
--builtin_ntp_servers=127.5.108.148:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:45031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:05:42.736127 2597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:42.736413 2597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:42.736586 2597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:42.743709 2597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:42.743811 2597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:42.743888 2597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:42.743973 2597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:42.744056 2597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:42.752929 2597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:45031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:45031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:42.754175 2597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:42.755978 2597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:42.767501 2603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:42.767457 2605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:42.767537 2602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:42.768669 2597 server_base.cc:1034] running on GCE node
I20260806 01:05:42.769409 2597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:42.770689 2597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:42.771911 2597 hybrid_clock.cc:648] HybridClock initialized: now 1785978342771877 us; error 48 us; skew 500 ppm
I20260806 01:05:42.772256 2597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:42.774489 2597 webserver.cc:460] Webserver started at http://127.5.108.190:45545/ using document root <none> and password file <none>
I20260806 01:05:42.775206 2597 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:42.775347 2597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:42.775682 2597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:42.778563 2597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "4dd1359ba68240bc879a852f7dff7d67"
format_stamp: "Formatted at 2026-08-06 01:05:42 on dist-test-slave-x4qh"
server_key: "55932a214beee2723b463c91b458dcdb"
server_key_iv: "1f57a81aa13224d2f67c9d21fe011602"
server_key_version: "encryptionkey@0"
I20260806 01:05:42.779361 2597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "4dd1359ba68240bc879a852f7dff7d67"
format_stamp: "Formatted at 2026-08-06 01:05:42 on dist-test-slave-x4qh"
server_key: "55932a214beee2723b463c91b458dcdb"
server_key_iv: "1f57a81aa13224d2f67c9d21fe011602"
server_key_version: "encryptionkey@0"
I20260806 01:05:42.784227 2597 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:05:42.787449 2614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:42.788740 2597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:05:42.788932 2597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "4dd1359ba68240bc879a852f7dff7d67"
format_stamp: "Formatted at 2026-08-06 01:05:42 on dist-test-slave-x4qh"
server_key: "55932a214beee2723b463c91b458dcdb"
server_key_iv: "1f57a81aa13224d2f67c9d21fe011602"
server_key_version: "encryptionkey@0"
I20260806 01:05:42.789141 2597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:42.813386 2597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:42.814160 2597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:42.814453 2597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:42.831646 2597 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:45031
I20260806 01:05:42.831665 2697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:45031 every 8 connection(s)
I20260806 01:05:42.832847 2597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:42.836156 2699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:42.839089 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 2597
I20260806 01:05:42.839267 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:42.839779 5554 external_mini_cluster.cc:1428] Setting key 7fb9000b61c4c858116c16bb9e72f6f1
I20260806 01:05:42.849062 2699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67: Bootstrap starting.
I20260806 01:05:42.852669 2699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:42.853893 2699 log.cc:824] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:42.854450 2703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:42.841557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.5.108.190:45031)
0806 01:05:42.842530 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:42.842545 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:42.842594 (+ 49us) server_negotiation.cc:407] Connection header received
0806 01:05:42.844603 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:42.844633 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:42.845204 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:42.845620 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:42.846090 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:42.848169 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:42.849577 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:42.850505 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:42.850706 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:42.851756 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:42.851766 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:42.851784 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:42.852424 (+ 640us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:42.852555 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:42.852594 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:42.852721 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:42.852976 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 01:05:42.853293 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":301,"threads_started":1}
I20260806 01:05:42.857030 2699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67: No bootstrap required, opened a new log
I20260806 01:05:42.863992 2699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd1359ba68240bc879a852f7dff7d67" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45031 } }
I20260806 01:05:42.864312 2699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:42.864464 2699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd1359ba68240bc879a852f7dff7d67, State: Initialized, Role: FOLLOWER
I20260806 01:05:42.864997 2699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd1359ba68240bc879a852f7dff7d67" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45031 } }
I20260806 01:05:42.865226 2699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:42.865372 2699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:42.865535 2699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:42.867601 2699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd1359ba68240bc879a852f7dff7d67" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45031 } }
I20260806 01:05:42.868076 2699 leader_election.cc:302] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dd1359ba68240bc879a852f7dff7d67; no voters:
I20260806 01:05:42.868666 2699 leader_election.cc:288] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:42.868824 2707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:42.869575 2707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [term 1 LEADER]: Becoming Leader. State: Replica: 4dd1359ba68240bc879a852f7dff7d67, State: Running, Role: LEADER
I20260806 01:05:42.870119 2707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd1359ba68240bc879a852f7dff7d67" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45031 } }
I20260806 01:05:42.870906 2699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:42.873337 2709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dd1359ba68240bc879a852f7dff7d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd1359ba68240bc879a852f7dff7d67" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45031 } } }
I20260806 01:05:42.873294 2710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dd1359ba68240bc879a852f7dff7d67. Latest consensus state: current_term: 1 leader_uuid: "4dd1359ba68240bc879a852f7dff7d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd1359ba68240bc879a852f7dff7d67" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 45031 } } }
I20260806 01:05:42.873785 2709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:42.873795 2710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:42.874725 2721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:42.881397 2721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:42.889930 2721 catalog_manager.cc:1347] Generated new cluster ID: f61dfa27ffdc4f248702858afb62f36f
I20260806 01:05:42.890031 2721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:42.904700 2721 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:42.906387 2721 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:42.914969 2721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67: Generated new TSK 0
I20260806 01:05:42.915917 2721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:05:42.926818 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.129:0
--local_ip_for_outbound_sockets=127.5.108.129
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:45031
--builtin_ntp_servers=127.5.108.148:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:05:43.266453 2734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:43.266700 2734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:43.266860 2734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:43.274113 2734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:43.274228 2734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:43.274396 2734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.129
I20260806 01:05:43.282819 2734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.108.129
--webserver_port=0
--tserver_master_addrs=127.5.108.190:45031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.129
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:43.283983 2734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:43.285774 2734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:43.299042 2742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:43.298983 2744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:43.298944 2747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:43.299960 2734 server_base.cc:1034] running on GCE node
I20260806 01:05:43.300534 2734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:43.301410 2734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:43.302611 2734 hybrid_clock.cc:648] HybridClock initialized: now 1785978343302572 us; error 36 us; skew 500 ppm
I20260806 01:05:43.302945 2734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:43.305256 2734 webserver.cc:460] Webserver started at http://127.5.108.129:40149/ using document root <none> and password file <none>
I20260806 01:05:43.305912 2734 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:43.306028 2734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:43.306363 2734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:43.309155 2734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/data/instance:
uuid: "18cdc05b0d3d464ab26c6b3007de9f9f"
format_stamp: "Formatted at 2026-08-06 01:05:43 on dist-test-slave-x4qh"
server_key: "4c973fa6b52858718bbfb0eea84c2d7b"
server_key_iv: "27b78b885a269d882d3ba4fda9a5c990"
server_key_version: "encryptionkey@0"
I20260806 01:05:43.309933 2734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance:
uuid: "18cdc05b0d3d464ab26c6b3007de9f9f"
format_stamp: "Formatted at 2026-08-06 01:05:43 on dist-test-slave-x4qh"
server_key: "4c973fa6b52858718bbfb0eea84c2d7b"
server_key_iv: "27b78b885a269d882d3ba4fda9a5c990"
server_key_version: "encryptionkey@0"
I20260806 01:05:43.314705 2734 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260806 01:05:43.318136 2755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:43.319537 2734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:43.319705 2734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/wal
uuid: "18cdc05b0d3d464ab26c6b3007de9f9f"
format_stamp: "Formatted at 2026-08-06 01:05:43 on dist-test-slave-x4qh"
server_key: "4c973fa6b52858718bbfb0eea84c2d7b"
server_key_iv: "27b78b885a269d882d3ba4fda9a5c990"
server_key_version: "encryptionkey@0"
I20260806 01:05:43.319911 2734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:43.342839 2734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:43.343609 2734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:43.343899 2734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:43.344813 2734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:43.346189 2734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:43.346306 2734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:43.346431 2734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:43.346525 2734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:43.379886 2734 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.129:38193
I20260806 01:05:43.379997 2913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.129:38193 every 8 connection(s)
I20260806 01:05:43.381158 2734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/data/info.pb
I20260806 01:05:43.388844 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 2734
I20260806 01:05:43.389030 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/wal/instance
I20260806 01:05:43.389464 5554 external_mini_cluster.cc:1428] Setting key 66bd158c9f02725ba1959ac482660751
I20260806 01:05:43.392248 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.130:0
--local_ip_for_outbound_sockets=127.5.108.130
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:45031
--builtin_ntp_servers=127.5.108.148:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:05:43.393932 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:43.383752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.129:58667 (local address 127.5.108.190:45031)
0806 01:05:43.384212 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:43.384218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:43.385417 (+ 1199us) server_negotiation.cc:407] Connection header received
0806 01:05:43.387064 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:43.387068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:43.387131 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:43.387222 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:43.388247 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:43.389043 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:43.390774 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:43.391300 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:43.391465 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:43.392915 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:43.392921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:43.392926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:43.392950 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:43.392989 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:43.393001 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:43.393094 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:43.393370 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 01:05:43.393477 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":222,"threads_started":1}
I20260806 01:05:43.394691 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:43.384426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45031 (local address 127.5.108.129:58667)
0806 01:05:43.385266 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:43.385315 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:43.386834 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:43.387472 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:43.387479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:43.387557 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:43.388095 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:43.388110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:43.389203 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:43.389207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:43.390622 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:43.390634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:43.391565 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:43.391574 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:43.391807 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:43.392591 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:05:43.392593 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:43.392635 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:05:43.392746 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:43.393096 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:43.393110 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:43.393118 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 01:05:43.393341 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 01:05:43.393643 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":273,"threads_started":1}
I20260806 01:05:43.396107 2915 heartbeater.cc:347] Connected to a master server at 127.5.108.190:45031
I20260806 01:05:43.396394 2915 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:43.397099 2915 heartbeater.cc:511] Master 127.5.108.190:45031 requested a full tablet report, sending...
I20260806 01:05:43.399001 2639 ts_manager.cc:194] Registered new tserver with Master: 18cdc05b0d3d464ab26c6b3007de9f9f (127.5.108.129:38193)
I20260806 01:05:43.400530 2639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.129:58667
W20260806 01:05:43.723474 2923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:43.723764 2923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:43.723934 2923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:43.730806 2923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:43.730930 2923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:43.731099 2923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.130
I20260806 01:05:43.739506 2923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.108.130
--webserver_port=0
--tserver_master_addrs=127.5.108.190:45031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.130
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:43.740682 2923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:43.742395 2923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:43.755020 2930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:43.755010 2933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:43.755082 2931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:43.755931 2923 server_base.cc:1034] running on GCE node
I20260806 01:05:43.756562 2923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:43.757486 2923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:43.758740 2923 hybrid_clock.cc:648] HybridClock initialized: now 1785978343758693 us; error 59 us; skew 500 ppm
I20260806 01:05:43.759058 2923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:43.761255 2923 webserver.cc:460] Webserver started at http://127.5.108.130:43053/ using document root <none> and password file <none>
I20260806 01:05:43.761886 2923 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:43.762035 2923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:43.762364 2923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:43.765296 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/data/instance:
uuid: "bf174943ab6948cb9ec6e6a4d1e71690"
format_stamp: "Formatted at 2026-08-06 01:05:43 on dist-test-slave-x4qh"
server_key: "0ce988a7c42091f8cdf2d6010603b4c4"
server_key_iv: "c9ac147636e9c5996606f1f8d892773f"
server_key_version: "encryptionkey@0"
I20260806 01:05:43.766127 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance:
uuid: "bf174943ab6948cb9ec6e6a4d1e71690"
format_stamp: "Formatted at 2026-08-06 01:05:43 on dist-test-slave-x4qh"
server_key: "0ce988a7c42091f8cdf2d6010603b4c4"
server_key_iv: "c9ac147636e9c5996606f1f8d892773f"
server_key_version: "encryptionkey@0"
I20260806 01:05:43.770916 2923 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260806 01:05:43.774511 2943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:43.775815 2923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:05:43.775982 2923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/wal
uuid: "bf174943ab6948cb9ec6e6a4d1e71690"
format_stamp: "Formatted at 2026-08-06 01:05:43 on dist-test-slave-x4qh"
server_key: "0ce988a7c42091f8cdf2d6010603b4c4"
server_key_iv: "c9ac147636e9c5996606f1f8d892773f"
server_key_version: "encryptionkey@0"
I20260806 01:05:43.776206 2923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:43.799933 2923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:43.800628 2923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:43.800918 2923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:43.801789 2923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:43.803117 2923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:43.803258 2923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:43.803354 2923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:43.803467 2923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 01:05:43.843940 2923 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.130:46229
I20260806 01:05:43.844038 3104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.130:46229 every 8 connection(s)
I20260806 01:05:43.845162 2923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/data/info.pb
I20260806 01:05:43.854257 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 2923
I20260806 01:05:43.854447 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/wal/instance
I20260806 01:05:43.854851 5554 external_mini_cluster.cc:1428] Setting key 26c3a28dee0abbd2e7d8fc2b2c299eee
I20260806 01:05:43.856364 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:43.847520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:59419 (local address 127.5.108.190:45031)
0806 01:05:43.847716 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:43.847722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:43.849002 (+ 1280us) server_negotiation.cc:407] Connection header received
0806 01:05:43.850618 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:43.850621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:43.850686 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:43.850783 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:43.851739 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:43.852386 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:43.853758 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:43.854292 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:43.854430 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:43.855692 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:43.855698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:43.855701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:43.855718 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:43.855754 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:43.855771 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:43.855852 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:43.856083 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 01:05:43.856160 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:05:43.857251 3111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:43.848030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45031 (local address 127.5.108.130:59419)
0806 01:05:43.848859 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:43.848899 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:43.850417 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:43.850955 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:43.850961 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:43.851035 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:43.851598 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:43.851615 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:43.852499 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:43.852502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:43.853646 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:43.853657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:43.854526 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:43.854532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:43.854707 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:43.855361 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:05:43.855364 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:43.855421 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:05:43.855529 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:43.855882 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:43.855892 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:43.855899 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:05:43.856056 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 01:05:43.856358 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":254,"threads_started":1}
I20260806 01:05:43.857597 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.108.131:0
--local_ip_for_outbound_sockets=127.5.108.131
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:45031
--builtin_ntp_servers=127.5.108.148:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:05:43.858456 3105 heartbeater.cc:347] Connected to a master server at 127.5.108.190:45031
I20260806 01:05:43.858777 3105 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:43.859512 3105 heartbeater.cc:511] Master 127.5.108.190:45031 requested a full tablet report, sending...
I20260806 01:05:43.860790 2639 ts_manager.cc:194] Registered new tserver with Master: bf174943ab6948cb9ec6e6a4d1e71690 (127.5.108.130:46229)
I20260806 01:05:43.861514 2639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.130:59419
W20260806 01:05:44.191610 3112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:44.191872 3112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:44.192028 3112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:44.198514 3112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:44.198647 3112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:44.198825 3112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.108.131
I20260806 01:05:44.207216 3112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.108.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.108.131
--webserver_port=0
--tserver_master_addrs=127.5.108.190:45031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.108.131
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:44.208353 3112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:44.210021 3112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:44.222359 3119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:44.222307 3122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:44.222309 3120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:44.222918 3112 server_base.cc:1034] running on GCE node
I20260806 01:05:44.223559 3112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:44.224467 3112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:44.225713 3112 hybrid_clock.cc:648] HybridClock initialized: now 1785978344225661 us; error 55 us; skew 500 ppm
I20260806 01:05:44.226015 3112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:05:44.228164 3112 webserver.cc:460] Webserver started at http://127.5.108.131:37297/ using document root <none> and password file <none>
I20260806 01:05:44.228814 3112 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:44.228929 3112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:44.229260 3112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:44.231935 3112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/data/instance:
uuid: "e749be60eacd4c21ae873dee594acd4b"
format_stamp: "Formatted at 2026-08-06 01:05:44 on dist-test-slave-x4qh"
server_key: "2573c619c3ee69ab1b4405304c907efc"
server_key_iv: "015d2883e15698d3b4d7f9eba49dfa5b"
server_key_version: "encryptionkey@0"
I20260806 01:05:44.232663 3112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance:
uuid: "e749be60eacd4c21ae873dee594acd4b"
format_stamp: "Formatted at 2026-08-06 01:05:44 on dist-test-slave-x4qh"
server_key: "2573c619c3ee69ab1b4405304c907efc"
server_key_iv: "015d2883e15698d3b4d7f9eba49dfa5b"
server_key_version: "encryptionkey@0"
I20260806 01:05:44.237303 3112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:05:44.240545 3129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:44.241729 3112 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 01:05:44.241894 3112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/wal
uuid: "e749be60eacd4c21ae873dee594acd4b"
format_stamp: "Formatted at 2026-08-06 01:05:44 on dist-test-slave-x4qh"
server_key: "2573c619c3ee69ab1b4405304c907efc"
server_key_iv: "015d2883e15698d3b4d7f9eba49dfa5b"
server_key_version: "encryptionkey@0"
I20260806 01:05:44.242121 3112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:44.262694 3112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:44.263415 3112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:44.263700 3112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:44.264602 3112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:05:44.266022 3112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:05:44.266156 3112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:44.266281 3112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:05:44.266378 3112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:44.299409 3112 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.131:34915
I20260806 01:05:44.299523 3288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.131:34915 every 8 connection(s)
I20260806 01:05:44.300637 3112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/data/info.pb
I20260806 01:05:44.309031 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 3112
I20260806 01:05:44.309206 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/wal/instance
I20260806 01:05:44.309680 5554 external_mini_cluster.cc:1428] Setting key 0f59ec33e9c44381316e2f1a66ba54d6
I20260806 01:05:44.312840 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.303043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.131:52357 (local address 127.5.108.190:45031)
0806 01:05:44.303228 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:44.303232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:44.304613 (+ 1381us) server_negotiation.cc:407] Connection header received
0806 01:05:44.306293 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:44.306298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:44.306367 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:44.306466 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:44.307447 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.308207 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.309856 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.310423 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.310560 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.312103 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:44.312109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:44.312113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:44.312134 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:44.312173 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:44.312192 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:44.312302 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:44.312545 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 01:05:44.312625 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:05:44.313684 3293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.303599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:45031 (local address 127.5.108.131:52357)
0806 01:05:44.304457 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:44.304491 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:44.306053 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:44.306654 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:44.306661 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:44.306752 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:05:44.307246 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.307261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.308349 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.308353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.309672 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.309684 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.310641 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.310648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.310994 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.311784 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:05:44.311787 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:05:44.311823 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:05:44.311896 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:05:44.312321 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:05:44.312332 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:05:44.312339 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:05:44.312519 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 01:05:44.312809 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":252,"threads_started":1}
I20260806 01:05:44.314860 3289 heartbeater.cc:347] Connected to a master server at 127.5.108.190:45031
I20260806 01:05:44.315145 3289 heartbeater.cc:464] Registering TS with master...
I20260806 01:05:44.315876 3289 heartbeater.cc:511] Master 127.5.108.190:45031 requested a full tablet report, sending...
I20260806 01:05:44.317154 2639 ts_manager.cc:194] Registered new tserver with Master: e749be60eacd4c21ae873dee594acd4b (127.5.108.131:34915)
I20260806 01:05:44.317901 2639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.108.131:52357
I20260806 01:05:44.324719 5554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:05:44.334223 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.328967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33246 (local address 127.5.108.190:45031)
0806 01:05:44.329151 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:44.329155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:44.329645 (+ 490us) server_negotiation.cc:407] Connection header received
0806 01:05:44.329793 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:44.329796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:44.329856 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:44.329930 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:44.330338 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.330962 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.332299 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.332681 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.332769 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.333712 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:44.333716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:44.333719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:44.333734 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:44.333762 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:44.333773 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:44.333840 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:44.333959 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 01:05:44.334022 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:05:44.337584 2639 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:05:44.343951 2639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:05:44.366887 3306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.355725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:34915 (local address 127.0.0.1:33816)
0806 01:05:44.358297 (+ 2572us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:44.358312 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:44.358448 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:44.359392 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:44.359397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:44.359424 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:44.359886 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.359907 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.361761 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.361766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.363789 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.363802 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.365986 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.365992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.366210 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.366264 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 01:05:44.366389 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 01:05:44.366579 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2440,"thread_start_us":525,"threads_started":1}
I20260806 01:05:44.368057 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.355927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.5.108.131:34915)
0806 01:05:44.357729 (+ 1802us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:44.357735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:44.358556 (+ 821us) server_negotiation.cc:407] Connection header received
0806 01:05:44.358675 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:44.358682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:44.358906 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:44.359091 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:44.360147 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.361592 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.363990 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.365453 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.365609 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.365852 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:44.367503 (+ 1651us) server_negotiation.cc:299] Negotiation successful
0806 01:05:44.367657 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1645,"thread_start_us":1511,"threads_started":1}
I20260806 01:05:44.368889 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.355072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38193 (local address 127.0.0.1:51232)
0806 01:05:44.356723 (+ 1651us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:44.356765 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:44.356980 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:44.357846 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:44.357855 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:44.357894 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:44.358301 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.358315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.361446 (+ 3131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.361451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.364564 (+ 3113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.364578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.368130 (+ 3552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.368140 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.368329 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.368359 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:05:44.368435 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 01:05:44.368592 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1339,"thread_start_us":2918,"threads_started":1}
I20260806 01:05:44.369645 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.355130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.5.108.129:38193)
0806 01:05:44.356072 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:44.356080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:44.357198 (+ 1118us) server_negotiation.cc:407] Connection header received
0806 01:05:44.357309 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:44.357318 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:44.357560 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:44.357744 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:44.358524 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.361292 (+ 2768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.364793 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.366473 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.366580 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.366783 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:44.369096 (+ 2313us) server_negotiation.cc:299] Negotiation successful
0806 01:05:44.369263 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":685,"threads_started":1}
I20260806 01:05:44.372448 3196 tablet_service.cc:1467] Processing CreateTablet for tablet 26b633ca278b41068c5ea7a39fe17625 (DEFAULT_TABLE table=test-table [id=937a1c3085224fe88bc1248c7b9a7b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:44.373989 3196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b633ca278b41068c5ea7a39fe17625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:44.373597 2814 tablet_service.cc:1467] Processing CreateTablet for tablet 26b633ca278b41068c5ea7a39fe17625 (DEFAULT_TABLE table=test-table [id=937a1c3085224fe88bc1248c7b9a7b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:44.374805 2814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b633ca278b41068c5ea7a39fe17625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:44.377707 3306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.358319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.130:46229 (local address 127.0.0.1:48278)
0806 01:05:44.367332 (+ 9013us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:44.367347 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:44.368337 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:44.371300 (+ 2963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:44.371306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:44.371335 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:44.371509 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.371520 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.373555 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.373563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.375250 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.375268 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.376985 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.376991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.377176 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.377210 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:05:44.377303 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 01:05:44.377432 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8929,"thread_start_us":11962,"threads_started":1}
I20260806 01:05:44.377986 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.358628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48278 (local address 127.5.108.130:46229)
0806 01:05:44.370771 (+ 12143us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:44.370779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:44.370806 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:05:44.370893 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:44.370899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:44.371094 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:44.371321 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:44.371662 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.373403 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.375438 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.376801 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.376950 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.377156 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:44.377359 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 01:05:44.377534 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11995,"thread_start_us":11879,"threads_started":1}
I20260806 01:05:44.382012 3008 tablet_service.cc:1467] Processing CreateTablet for tablet 26b633ca278b41068c5ea7a39fe17625 (DEFAULT_TABLE table=test-table [id=937a1c3085224fe88bc1248c7b9a7b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:05:44.383198 3008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b633ca278b41068c5ea7a39fe17625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:44.391542 3315 tablet_bootstrap.cc:492] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: Bootstrap starting.
I20260806 01:05:44.396642 3315 tablet_bootstrap.cc:654] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:44.397504 3314 tablet_bootstrap.cc:492] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: Bootstrap starting.
I20260806 01:05:44.398453 3315 log.cc:824] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:44.401028 3314 tablet_bootstrap.cc:654] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:44.401364 3315 tablet_bootstrap.cc:492] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: No bootstrap required, opened a new log
I20260806 01:05:44.401612 3315 ts_tablet_manager.cc:1397] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260806 01:05:44.402009 3314 log.cc:824] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:44.403187 2915 heartbeater.cc:503] Master 127.5.108.190:45031 was elected leader, sending a full tablet report...
I20260806 01:05:44.403470 3316 tablet_bootstrap.cc:492] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: Bootstrap starting.
I20260806 01:05:44.406512 3314 tablet_bootstrap.cc:492] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: No bootstrap required, opened a new log
I20260806 01:05:44.406821 3314 ts_tablet_manager.cc:1397] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 01:05:44.407073 3316 tablet_bootstrap.cc:654] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:44.408340 3316 log.cc:824] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:44.410599 3316 tablet_bootstrap.cc:492] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: No bootstrap required, opened a new log
I20260806 01:05:44.410835 3316 ts_tablet_manager.cc:1397] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260806 01:05:44.411623 3315 raft_consensus.cc:348] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } }
I20260806 01:05:44.412009 3315 raft_consensus.cc:374] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:44.412114 3315 raft_consensus.cc:729] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18cdc05b0d3d464ab26c6b3007de9f9f, State: Initialized, Role: FOLLOWER
I20260806 01:05:44.412664 3315 consensus_queue.cc:260] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } }
I20260806 01:05:44.414186 3315 ts_tablet_manager.cc:1428] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260806 01:05:44.416330 3314 raft_consensus.cc:348] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } }
I20260806 01:05:44.416774 3314 raft_consensus.cc:374] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:44.416880 3314 raft_consensus.cc:729] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e749be60eacd4c21ae873dee594acd4b, State: Initialized, Role: FOLLOWER
I20260806 01:05:44.417407 3314 consensus_queue.cc:260] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } }
I20260806 01:05:44.418355 3289 heartbeater.cc:503] Master 127.5.108.190:45031 was elected leader, sending a full tablet report...
I20260806 01:05:44.418908 3314 ts_tablet_manager.cc:1428] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: Time spent starting tablet: real 0.012s user 0.003s sys 0.008s
I20260806 01:05:44.419071 3316 raft_consensus.cc:348] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } }
I20260806 01:05:44.419412 3316 raft_consensus.cc:374] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:44.419533 3316 raft_consensus.cc:729] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf174943ab6948cb9ec6e6a4d1e71690, State: Initialized, Role: FOLLOWER
I20260806 01:05:44.420137 3316 consensus_queue.cc:260] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } }
I20260806 01:05:44.421195 3105 heartbeater.cc:503] Master 127.5.108.190:45031 was elected leader, sending a full tablet report...
I20260806 01:05:44.421731 3316 ts_tablet_manager.cc:1428] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
W20260806 01:05:44.552304 3290 tablet.cc:2369] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:05:44.597287 3106 tablet.cc:2369] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:05:44.633642 2916 tablet.cc:2369] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:05:44.727208 3324 raft_consensus.cc:482] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:05:44.727552 3324 raft_consensus.cc:504] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } }
I20260806 01:05:44.729177 3324 leader_election.cc:288] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18cdc05b0d3d464ab26c6b3007de9f9f (127.5.108.129:38193), e749be60eacd4c21ae873dee594acd4b (127.5.108.131:34915)
I20260806 01:05:44.736287 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.729631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:51757 (local address 127.5.108.129:38193)
0806 01:05:44.729848 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:44.729854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:44.730625 (+ 771us) server_negotiation.cc:407] Connection header received
0806 01:05:44.730809 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:44.730813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:44.730897 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:44.730981 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:44.731823 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.732447 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.734444 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.735298 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.735483 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.735571 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:44.735894 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 01:05:44.736028 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:05:44.736316 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.729753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.108.130:56085 (local address 127.5.108.131:34915)
0806 01:05:44.729966 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:44.729973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:44.730644 (+ 671us) server_negotiation.cc:407] Connection header received
0806 01:05:44.730809 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:44.730813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:44.730890 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:44.730982 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:05:44.731774 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.732445 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.734499 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.735313 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.735535 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.735603 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:44.735957 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 01:05:44.736079 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:05:44.736398 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.729474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.129:38193 (local address 127.5.108.130:51757)
0806 01:05:44.730465 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:44.730484 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:44.730669 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:44.731013 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:44.731018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:44.731049 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:44.731644 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.731659 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.732593 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.732596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.734249 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.734262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.735536 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.735542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.735723 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.735747 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:05:44.735821 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 01:05:44.735972 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":166,"spinlock_wait_cycles":27648,"thread_start_us":736,"threads_started":1}
I20260806 01:05:44.736645 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.729606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.131:34915 (local address 127.5.108.130:56085)
0806 01:05:44.730522 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 01:05:44.730538 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:05:44.730669 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:05:44.731013 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:05:44.731018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:05:44.731047 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:05:44.731600 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.731612 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.732593 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.732597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.734250 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:05:44.734263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.735578 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.735588 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:05:44.735741 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.735765 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:05:44.735873 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 01:05:44.735994 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":147,"spinlock_wait_cycles":10368,"thread_start_us":485,"threads_started":1}
I20260806 01:05:44.737386 2841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b633ca278b41068c5ea7a39fe17625" candidate_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" is_pre_election: true
I20260806 01:05:44.737562 3225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b633ca278b41068c5ea7a39fe17625" candidate_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e749be60eacd4c21ae873dee594acd4b" is_pre_election: true
I20260806 01:05:44.737968 2841 raft_consensus.cc:2454] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf174943ab6948cb9ec6e6a4d1e71690 in term 0.
I20260806 01:05:44.738130 3225 raft_consensus.cc:2454] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf174943ab6948cb9ec6e6a4d1e71690 in term 0.
I20260806 01:05:44.738648 2950 leader_election.cc:302] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18cdc05b0d3d464ab26c6b3007de9f9f, bf174943ab6948cb9ec6e6a4d1e71690; no voters:
I20260806 01:05:44.739049 3324 raft_consensus.cc:2781] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:05:44.739169 3324 raft_consensus.cc:482] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:05:44.739295 3324 raft_consensus.cc:3037] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:44.741396 3324 raft_consensus.cc:504] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } }
I20260806 01:05:44.741911 3324 leader_election.cc:288] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [CANDIDATE]: Term 1 election: Requested vote from peers 18cdc05b0d3d464ab26c6b3007de9f9f (127.5.108.129:38193), e749be60eacd4c21ae873dee594acd4b (127.5.108.131:34915)
I20260806 01:05:44.742357 3225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b633ca278b41068c5ea7a39fe17625" candidate_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e749be60eacd4c21ae873dee594acd4b"
I20260806 01:05:44.742357 2841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b633ca278b41068c5ea7a39fe17625" candidate_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f"
I20260806 01:05:44.742544 3225 raft_consensus.cc:3037] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:44.742544 2841 raft_consensus.cc:3037] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:44.745682 3225 raft_consensus.cc:2454] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf174943ab6948cb9ec6e6a4d1e71690 in term 1.
I20260806 01:05:44.745682 2841 raft_consensus.cc:2454] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf174943ab6948cb9ec6e6a4d1e71690 in term 1.
I20260806 01:05:44.746129 2946 leader_election.cc:302] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf174943ab6948cb9ec6e6a4d1e71690, e749be60eacd4c21ae873dee594acd4b; no voters:
I20260806 01:05:44.746443 3324 raft_consensus.cc:2781] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:44.746906 3324 raft_consensus.cc:686] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 1 LEADER]: Becoming Leader. State: Replica: bf174943ab6948cb9ec6e6a4d1e71690, State: Running, Role: LEADER
I20260806 01:05:44.747341 3324 consensus_queue.cc:237] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } }
I20260806 01:05:44.751806 2639 catalog_manager.cc:5495] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf174943ab6948cb9ec6e6a4d1e71690 (127.5.108.130). New cstate: current_term: 1 leader_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf174943ab6948cb9ec6e6a4d1e71690" member_type: VOTER last_known_addr { host: "127.5.108.130" port: 46229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 } health_report { overall_health: UNKNOWN } } }
I20260806 01:05:44.819458 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.814580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.5.108.130:46229)
0806 01:05:44.814777 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:44.814782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:44.814798 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:05:44.814886 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:44.814890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:44.814996 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:44.815113 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:05:44.815494 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.816189 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.817471 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.817856 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.817942 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.818327 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:05:44.818844 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:05:44.818948 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:44.819048 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 01:05:44.819107 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:05:44.827539 3225 raft_consensus.cc:1261] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [term 1 FOLLOWER]: Refusing update from remote peer bf174943ab6948cb9ec6e6a4d1e71690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:44.827543 2841 raft_consensus.cc:1261] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [term 1 FOLLOWER]: Refusing update from remote peer bf174943ab6948cb9ec6e6a4d1e71690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:05:44.828646 3324 consensus_queue.cc:1035] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e749be60eacd4c21ae873dee594acd4b" member_type: VOTER last_known_addr { host: "127.5.108.131" port: 34915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:44.829011 3328 consensus_queue.cc:1035] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18cdc05b0d3d464ab26c6b3007de9f9f" member_type: VOTER last_known_addr { host: "127.5.108.129" port: 38193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:05:44.846568 3336 mvcc.cc:205] Tried to move back new op lower bound from 7315367300403146752 to 7315367300089004032. Current Snapshot: MvccSnapshot[applied={T|T < 7315367300403146752}]
I20260806 01:05:44.873291 2639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:44.873588 2639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:05:44.877332 2639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dd1359ba68240bc879a852f7dff7d67: Sending DeleteTablet for 3 replicas of tablet 26b633ca278b41068c5ea7a39fe17625
I20260806 01:05:44.878424 2814 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b633ca278b41068c5ea7a39fe17625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:44 UTC) from {username='slave'} at 127.0.0.1:51232
I20260806 01:05:44.878522 3008 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b633ca278b41068c5ea7a39fe17625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:44 UTC) from {username='slave'} at 127.0.0.1:48278
I20260806 01:05:44.879477 3344 tablet_replica.cc:330] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: stopping tablet replica
I20260806 01:05:44.879451 3196 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b633ca278b41068c5ea7a39fe17625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:05:44 UTC) from {username='slave'} at 127.0.0.1:33816
I20260806 01:05:44.880010 3344 raft_consensus.cc:2229] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:44.880720 3344 raft_consensus.cc:2258] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:44.881263 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 2734
I20260806 01:05:44.883230 3344 ts_tablet_manager.cc:1905] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:44.892719 3344 ts_tablet_manager.cc:1918] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:44.893047 3344 log.cc:1196] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-0/wal/wals/26b633ca278b41068c5ea7a39fe17625
I20260806 01:05:44.893689 3344 ts_tablet_manager.cc:1939] T 26b633ca278b41068c5ea7a39fe17625 P 18cdc05b0d3d464ab26c6b3007de9f9f: Deleting consensus metadata
I20260806 01:05:44.893903 3345 tablet_replica.cc:330] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: stopping tablet replica
I20260806 01:05:44.894418 3345 raft_consensus.cc:2229] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:05:44.895103 3345 raft_consensus.cc:2258] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:44.895618 2618 catalog_manager.cc:4841] TS 18cdc05b0d3d464ab26c6b3007de9f9f (127.5.108.129:38193): tablet 26b633ca278b41068c5ea7a39fe17625 (table test-table [id=937a1c3085224fe88bc1248c7b9a7b54]) successfully deleted
I20260806 01:05:44.896919 3345 ts_tablet_manager.cc:1905] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:44.900594 3346 tablet_replica.cc:330] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: stopping tablet replica
I20260806 01:05:44.901043 3346 raft_consensus.cc:2229] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:05:44.901561 3346 raft_consensus.cc:2258] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:05:44.903517 3345 ts_tablet_manager.cc:1918] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:44.903761 3345 log.cc:1196] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-1/wal/wals/26b633ca278b41068c5ea7a39fe17625
I20260806 01:05:44.903774 3346 ts_tablet_manager.cc:1905] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:05:44.904443 3345 ts_tablet_manager.cc:1939] T 26b633ca278b41068c5ea7a39fe17625 P bf174943ab6948cb9ec6e6a4d1e71690: Deleting consensus metadata
I20260806 01:05:44.906685 2618 catalog_manager.cc:4841] TS bf174943ab6948cb9ec6e6a4d1e71690 (127.5.108.130:46229): tablet 26b633ca278b41068c5ea7a39fe17625 (table test-table [id=937a1c3085224fe88bc1248c7b9a7b54]) successfully deleted
I20260806 01:05:44.909847 3346 ts_tablet_manager.cc:1918] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:05:44.910027 3346 log.cc:1196] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: Deleting WAL directory at /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978022745104-5554-0/minicluster-data/ts-2/wal/wals/26b633ca278b41068c5ea7a39fe17625
I20260806 01:05:44.910578 3346 ts_tablet_manager.cc:1939] T 26b633ca278b41068c5ea7a39fe17625 P e749be60eacd4c21ae873dee594acd4b: Deleting consensus metadata
I20260806 01:05:44.912293 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:44.882721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51246 (local address 127.5.108.129:38193)
0806 01:05:44.882894 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:44.882900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:44.899957 (+ 17057us) server_negotiation.cc:407] Connection header received
0806 01:05:44.900147 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:44.900153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:44.900255 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:44.900365 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:44.901012 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.902078 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.904041 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:44.904742 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:44.904862 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:44.910492 (+ 5630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:44.910506 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:44.910526 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:44.911364 (+ 838us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:44.911454 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:44.911480 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:44.911620 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:44.911797 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 01:05:44.911925 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:05:44.912532 2616 catalog_manager.cc:4841] TS e749be60eacd4c21ae873dee594acd4b (127.5.108.131:34915): tablet 26b633ca278b41068c5ea7a39fe17625 (table test-table [id=937a1c3085224fe88bc1248c7b9a7b54]) successfully deleted
I20260806 01:05:44.912961 2907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:45.001399 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 2734
I20260806 01:05:45.029760 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 2923
I20260806 01:05:45.037514 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:45.030666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48292 (local address 127.5.108.130:46229)
0806 01:05:45.030850 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:45.030854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:45.030873 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:05:45.031043 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:45.031047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:45.031146 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:45.031234 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:45.031738 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:45.032499 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:45.034349 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:45.034780 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:45.034869 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:45.036075 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:45.036102 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:45.036117 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:45.036743 (+ 626us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:45.036789 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:45.036808 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:45.036897 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:45.037062 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 01:05:45.037150 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:05:45.038311 3100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:45.144436 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 2923
I20260806 01:05:45.169991 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 3112
I20260806 01:05:45.176797 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:45.170800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.5.108.131:34915)
0806 01:05:45.170976 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:45.170980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:45.170999 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:05:45.171144 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:45.171148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:45.171227 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:45.171321 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:45.171801 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:45.172494 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:45.173772 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:45.174185 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:45.174343 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:45.175357 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:45.175534 (+ 177us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:45.175549 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:45.176086 (+ 537us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:45.176130 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:45.176149 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:45.176275 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:45.176408 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 01:05:45.176509 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:05:45.177402 3281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:45.262935 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 3112
I20260806 01:05:45.284693 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 2597
I20260806 01:05:45.285851 2690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:45.388556 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 2597
2026-08-06T01:05:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3071 ms)
[----------] 6 tests from EncryptionPolicyTest (22414 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:45 dist-test-slave-x4qh krb5kdc[3359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:45 dist-test-slave-x4qh krb5kdc[3359](info): set up 2 sockets
Aug 06 01:05:45 dist-test-slave-x4qh krb5kdc[3359](info): commencing operation
krb5kdc: starting...
W20260806 01:05:46.645468 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:46 dist-test-slave-x4qh krb5kdc[3359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:05:46.948524 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:47.282127 3376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:47.282418 3376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:47.282570 3376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:47.289232 3376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:47.289372 3376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:47.289474 3376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:47.289578 3376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:47.289656 3376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:47.298681 3376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:47.299917 3376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:47.301165 3376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:47.312841 3382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:47.313045 3383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:47.312919 3386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:47.313946 3376 server_base.cc:1034] running on GCE node
I20260806 01:05:47.314687 3376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:47.316068 3376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:47.317297 3376 hybrid_clock.cc:648] HybridClock initialized: now 1785978347317264 us; error 48 us; skew 500 ppm
I20260806 01:05:47.317660 3376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:47 dist-test-slave-x4qh krb5kdc[3359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978347, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:47.320876 3376 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:05:47.322525 3376 webserver.cc:460] Webserver started at http://127.0.0.1:37479/ using document root <none> and password file <none>
I20260806 01:05:47.323432 3376 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:47.323577 3376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:47.323894 3376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:47.326804 3376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "60f90c3121f74b558e1f1abc7fcc42a0"
format_stamp: "Formatted at 2026-08-06 01:05:47 on dist-test-slave-x4qh"
server_key: "8a6592507a8844be22e9bca71f4141bb"
server_key_iv: "89f38f47c37cdafeb8f568ee3b636eaf"
server_key_version: "encryptionkey@0"
I20260806 01:05:47.327600 3376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "60f90c3121f74b558e1f1abc7fcc42a0"
format_stamp: "Formatted at 2026-08-06 01:05:47 on dist-test-slave-x4qh"
server_key: "8a6592507a8844be22e9bca71f4141bb"
server_key_iv: "89f38f47c37cdafeb8f568ee3b636eaf"
server_key_version: "encryptionkey@0"
I20260806 01:05:47.332489 3376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 01:05:47.335721 3399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:47.336903 3376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:05:47.337080 3376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "60f90c3121f74b558e1f1abc7fcc42a0"
format_stamp: "Formatted at 2026-08-06 01:05:47 on dist-test-slave-x4qh"
server_key: "8a6592507a8844be22e9bca71f4141bb"
server_key_iv: "89f38f47c37cdafeb8f568ee3b636eaf"
server_key_version: "encryptionkey@0"
I20260806 01:05:47.337265 3376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:47.363346 3376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:47.367111 3376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:47.367406 3376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:47.385569 3376 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37593
I20260806 01:05:47.385653 3474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37593 every 8 connection(s)
I20260806 01:05:47.386866 3376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:47.388249 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 3376
I20260806 01:05:47.388482 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:47.389348 5554 external_mini_cluster.cc:1428] Setting key a04fb87a50a26e9408c3968d356b6b91
I20260806 01:05:47.391544 3475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:05:47 dist-test-slave-x4qh krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:05:47.404655 3475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0: Bootstrap starting.
I20260806 01:05:47.407935 3475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:47.408994 3475 log.cc:824] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:47.412057 3475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0: No bootstrap required, opened a new log
I20260806 01:05:47.413116 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:47.391285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42087 (local address 127.0.0.1:37593)
0806 01:05:47.392200 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:47.392213 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:47.392258 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:05:47.394491 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:47.394517 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:47.395009 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:47.395474 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:47.396751 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:47.398820 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:47.400184 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:47.401195 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:47.401377 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:47.405911 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:47.405918 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:47.405929 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:47.405967 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:47.408992 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:47.410000 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:47.410012 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:47.410036 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:47.410260 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:47.410752 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:47.410755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:47.410757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:47.411197 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:47.411476 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:47.411931 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 01:05:47.412189 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":372,"threads_started":1}
I20260806 01:05:47.416828 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:05:47.419725 3475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f90c3121f74b558e1f1abc7fcc42a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37593 } }
I20260806 01:05:47.420099 3475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:47.420219 3475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60f90c3121f74b558e1f1abc7fcc42a0, State: Initialized, Role: FOLLOWER
I20260806 01:05:47.420712 3475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f90c3121f74b558e1f1abc7fcc42a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37593 } }
I20260806 01:05:47.420929 3475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:47.421063 3475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:47.421283 3475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:47.423640 3475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f90c3121f74b558e1f1abc7fcc42a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37593 } }
I20260806 01:05:47.424140 3475 leader_election.cc:302] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60f90c3121f74b558e1f1abc7fcc42a0; no voters:
I20260806 01:05:47.424810 3475 leader_election.cc:288] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:47.424924 3491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:47.427249 3475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:47.431665 3491 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [term 1 LEADER]: Becoming Leader. State: Replica: 60f90c3121f74b558e1f1abc7fcc42a0, State: Running, Role: LEADER
I20260806 01:05:47.432307 3491 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f90c3121f74b558e1f1abc7fcc42a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37593 } }
I20260806 01:05:47.435523 3492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60f90c3121f74b558e1f1abc7fcc42a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f90c3121f74b558e1f1abc7fcc42a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37593 } } }
I20260806 01:05:47.435916 3492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:47.437024 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:47.421530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39781 (local address 127.0.0.1:37593)
0806 01:05:47.421669 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:47.421675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:47.422125 (+ 450us) server_negotiation.cc:407] Connection header received
0806 01:05:47.422302 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:47.422311 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:47.422380 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:47.422484 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:47.423974 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:47.424853 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:47.426541 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:47.427063 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:47.427180 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:47.429963 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:47.429969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:47.429973 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:47.430039 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:47.432896 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:47.433650 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:47.433655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:47.433658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:47.433784 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:47.434222 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:47.434228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:47.434234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:47.434630 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:47.434966 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:47.435610 (+ 644us) server_negotiation.cc:299] Negotiation successful
0806 01:05:47.435830 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 01:05:47.437023 3494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60f90c3121f74b558e1f1abc7fcc42a0. Latest consensus state: current_term: 1 leader_uuid: "60f90c3121f74b558e1f1abc7fcc42a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f90c3121f74b558e1f1abc7fcc42a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37593 } } }
I20260806 01:05:47.439477 3494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:47.440464 3505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:47.444548 3505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:47.451562 3505 catalog_manager.cc:1347] Generated new cluster ID: cda3c7a31e344fe7a3225aea571618b0
I20260806 01:05:47.451659 3505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:47.505450 3505 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:47.507314 3505 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:47.517808 3505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60f90c3121f74b558e1f1abc7fcc42a0: Generated new TSK 0
I20260806 01:05:47.518669 3505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:05:47.526240 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 3376
I20260806 01:05:47.527402 3469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:47.634299 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 3376
W20260806 01:05:47.661034 3357 connection.cc:535] client connection to 127.0.0.1:37593 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37593): Connection reset by peer (error 104)
2026-08-06T01:05:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2239 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:47 dist-test-slave-x4qh krb5kdc[3524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:47 dist-test-slave-x4qh krb5kdc[3524](info): set up 2 sockets
Aug 06 01:05:47 dist-test-slave-x4qh krb5kdc[3524](info): commencing operation
krb5kdc: starting...
W20260806 01:05:48.883769 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:49 dist-test-slave-x4qh krb5kdc[3524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:05:49.157833 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45219
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:49.481791 3543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:49.482090 3543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:49.482252 3543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:49.489082 3543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:49.489219 3543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:49.489326 3543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:49.489434 3543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:49.489517 3543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:49.498618 3543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45219
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:49.499866 3543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:49.501094 3543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:49.513722 3553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:49.514622 3556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:49.515113 3543 server_base.cc:1034] running on GCE node
W20260806 01:05:49.514706 3550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:49.516279 3543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:49.518406 3543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:49.519678 3543 hybrid_clock.cc:648] HybridClock initialized: now 1785978349519560 us; error 87 us; skew 500 ppm
I20260806 01:05:49.520092 3543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:49 dist-test-slave-x4qh krb5kdc[3524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978349, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:49.523761 3543 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:05:49.525045 3543 webserver.cc:460] Webserver started at http://127.0.0.1:42787/ using document root <none> and password file <none>
I20260806 01:05:49.525633 3543 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:49.525725 3543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:49.525993 3543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:49.528720 3543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "70f94145b2bc4fcbb80475a57d055a31"
format_stamp: "Formatted at 2026-08-06 01:05:49 on dist-test-slave-x4qh"
server_key: "c7c178ae4d3b0c006190e7745d3738f0"
server_key_iv: "23664588d314dceea5042a829371186c"
server_key_version: "encryptionkey@0"
I20260806 01:05:49.529439 3543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "70f94145b2bc4fcbb80475a57d055a31"
format_stamp: "Formatted at 2026-08-06 01:05:49 on dist-test-slave-x4qh"
server_key: "c7c178ae4d3b0c006190e7745d3738f0"
server_key_iv: "23664588d314dceea5042a829371186c"
server_key_version: "encryptionkey@0"
I20260806 01:05:49.534152 3543 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 01:05:49.537088 3564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:49.538254 3543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 01:05:49.538424 3543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "70f94145b2bc4fcbb80475a57d055a31"
format_stamp: "Formatted at 2026-08-06 01:05:49 on dist-test-slave-x4qh"
server_key: "c7c178ae4d3b0c006190e7745d3738f0"
server_key_iv: "23664588d314dceea5042a829371186c"
server_key_version: "encryptionkey@0"
I20260806 01:05:49.538626 3543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:49.558223 3543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:49.562271 3543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:49.562561 3543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:49.580302 3543 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45219
I20260806 01:05:49.580348 3650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45219 every 8 connection(s)
I20260806 01:05:49.581553 3543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:49.583784 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 3543
I20260806 01:05:49.583962 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:49.584534 5554 external_mini_cluster.cc:1428] Setting key edeb52846711262a4bbacd5e771d12da
I20260806 01:05:49.585636 3651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:49.597216 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:49.586222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33633 (local address 127.0.0.1:45219)
0806 01:05:49.586933 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:49.586942 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:49.586980 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:05:49.588637 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:49.588657 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:49.589004 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:49.589342 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:49.589903 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:49.591188 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:49.593062 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:49.593652 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:49.593784 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:49.594973 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:49.594984 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:49.594998 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:49.595599 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:49.595675 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:49.595700 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:49.595859 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:49.596174 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 01:05:49.596365 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":320,"threads_started":1}
I20260806 01:05:49.598641 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31: Bootstrap starting.
I20260806 01:05:49.600886 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:05:49.601847 3651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:49.602759 3651 log.cc:824] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:49.605564 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31: No bootstrap required, opened a new log
I20260806 01:05:49.612010 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:49.605146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47763 (local address 127.0.0.1:45219)
0806 01:05:49.605338 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:49.605344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:49.606167 (+ 823us) server_negotiation.cc:407] Connection header received
0806 01:05:49.606329 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:49.606333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:49.606380 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:49.606452 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:49.606999 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:49.607752 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:49.609628 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:49.610060 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:49.610148 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:49.611257 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:49.611260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:49.611263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:49.611277 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:49.611322 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:49.611339 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:49.611468 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:49.611635 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 01:05:49.611671 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:05:49.614552 3651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f94145b2bc4fcbb80475a57d055a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } }
I20260806 01:05:49.614892 3651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:49.614987 3651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70f94145b2bc4fcbb80475a57d055a31, State: Initialized, Role: FOLLOWER
I20260806 01:05:49.615553 3651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f94145b2bc4fcbb80475a57d055a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } }
I20260806 01:05:49.615784 3651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:49.615938 3651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:49.616137 3651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:49.618093 3651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f94145b2bc4fcbb80475a57d055a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } }
I20260806 01:05:49.618547 3651 leader_election.cc:302] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70f94145b2bc4fcbb80475a57d055a31; no voters:
I20260806 01:05:49.619091 3651 leader_election.cc:288] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:49.619243 3668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:49.620076 3668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [term 1 LEADER]: Becoming Leader. State: Replica: 70f94145b2bc4fcbb80475a57d055a31, State: Running, Role: LEADER
I20260806 01:05:49.620584 3668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f94145b2bc4fcbb80475a57d055a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } }
I20260806 01:05:49.621199 3651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:49.623450 3670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70f94145b2bc4fcbb80475a57d055a31. Latest consensus state: current_term: 1 leader_uuid: "70f94145b2bc4fcbb80475a57d055a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f94145b2bc4fcbb80475a57d055a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } } }
I20260806 01:05:49.623952 3670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:49.623898 3669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70f94145b2bc4fcbb80475a57d055a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f94145b2bc4fcbb80475a57d055a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } } }
I20260806 01:05:49.624261 3669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:49.625931 3681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:49.630690 3681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:49.638314 3681 catalog_manager.cc:1347] Generated new cluster ID: e779df8bb017438392424f7218b4dc6a
I20260806 01:05:49.638414 3681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:49.656883 3681 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:49.658028 3681 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:49.664618 3681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70f94145b2bc4fcbb80475a57d055a31: Generated new TSK 0
I20260806 01:05:49.665308 3681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:05:49.674636 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 3543
W20260806 01:05:49.675719 3646 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33633
W20260806 01:05:49.676484 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 3543: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:05:49.676668 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 3543
W20260806 01:05:49.703100 3522 connection.cc:535] client connection to 127.0.0.1:45219 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45219): Connection reset by peer (error 104)
2026-08-06T01:05:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2041 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:49 dist-test-slave-x4qh krb5kdc[3698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:49 dist-test-slave-x4qh krb5kdc[3698](info): set up 2 sockets
Aug 06 01:05:49 dist-test-slave-x4qh krb5kdc[3698](info): commencing operation
krb5kdc: starting...
W20260806 01:05:50.947508 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:51 dist-test-slave-x4qh krb5kdc[3698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:05:51.234295 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:51.557696 3717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:51.557960 3717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:51.558048 3717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:51.564973 3717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:51.565096 3717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:51.565196 3717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:51.565299 3717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:51.565371 3717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:51.574280 3717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42739
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:51.575441 3717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:51.576697 3717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:51.588228 3723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:51.588238 3724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:51.588351 3726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:51.589761 3717 server_base.cc:1034] running on GCE node
I20260806 01:05:51.590459 3717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:51.591697 3717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:51.592909 3717 hybrid_clock.cc:648] HybridClock initialized: now 1785978351592872 us; error 44 us; skew 500 ppm
I20260806 01:05:51.593210 3717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:51 dist-test-slave-x4qh krb5kdc[3698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978351, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:51.600265 3717 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:05:51.601573 3717 webserver.cc:460] Webserver started at http://127.0.0.1:45361/ using document root <none> and password file <none>
I20260806 01:05:51.602221 3717 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:51.602350 3717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:51.602622 3717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:51.605387 3717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "4ae1481b55f64c638ec46a5d0ff7f51d"
format_stamp: "Formatted at 2026-08-06 01:05:51 on dist-test-slave-x4qh"
server_key: "b2a5e6393d7e9b6909188c9017532169"
server_key_iv: "25732e2b8a84f93eb5715e9675f0e4f6"
server_key_version: "encryptionkey@0"
I20260806 01:05:51.606189 3717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "4ae1481b55f64c638ec46a5d0ff7f51d"
format_stamp: "Formatted at 2026-08-06 01:05:51 on dist-test-slave-x4qh"
server_key: "b2a5e6393d7e9b6909188c9017532169"
server_key_iv: "25732e2b8a84f93eb5715e9675f0e4f6"
server_key_version: "encryptionkey@0"
I20260806 01:05:51.610898 3717 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260806 01:05:51.614126 3735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:51.615229 3717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:05:51.615428 3717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "4ae1481b55f64c638ec46a5d0ff7f51d"
format_stamp: "Formatted at 2026-08-06 01:05:51 on dist-test-slave-x4qh"
server_key: "b2a5e6393d7e9b6909188c9017532169"
server_key_iv: "25732e2b8a84f93eb5715e9675f0e4f6"
server_key_version: "encryptionkey@0"
I20260806 01:05:51.615608 3717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:51.621243 3717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:51.624641 3717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:51.624920 3717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:51.642307 3717 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42739
I20260806 01:05:51.642382 3820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42739 every 8 connection(s)
I20260806 01:05:51.643653 3717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:51.646878 3822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:51.649748 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 3717
I20260806 01:05:51.649941 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:51.650370 5554 external_mini_cluster.cc:1428] Setting key 988fcc131754b1432332a6ba3d790b43
I20260806 01:05:51.657562 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:51.651999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52343 (local address 127.0.0.1:42739)
0806 01:05:51.652730 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:51.652747 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:51.652785 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:05:51.654403 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:51.654422 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:51.654728 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:51.655037 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:51.656161 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:51.656167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:51.656178 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:51.656192 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:51.656254 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:51.656273 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:51.656426 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:51.656577 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 01:05:51.656758 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":348,"threads_started":1}
I20260806 01:05:51.658321 3822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d: Bootstrap starting.
I20260806 01:05:51.661037 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:05:51.661613 3822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:51.662598 3822 log.cc:824] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:51.665525 3822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d: No bootstrap required, opened a new log
I20260806 01:05:51.668088 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:51.665641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52551 (local address 127.0.0.1:42739)
0806 01:05:51.665791 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:51.665798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:51.666193 (+ 395us) server_negotiation.cc:407] Connection header received
0806 01:05:51.666372 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:51.666381 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:51.666423 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:51.666544 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:51.667429 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:51.667434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:51.667438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:51.667455 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:51.667501 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:51.667519 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:51.667649 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:51.667736 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 01:05:51.667795 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 01:05:51.673050 3822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae1481b55f64c638ec46a5d0ff7f51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } }
I20260806 01:05:51.673429 3822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:51.673542 3822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae1481b55f64c638ec46a5d0ff7f51d, State: Initialized, Role: FOLLOWER
I20260806 01:05:51.674153 3822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae1481b55f64c638ec46a5d0ff7f51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } }
I20260806 01:05:51.674408 3822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:51.674562 3822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:51.674731 3822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:51.676738 3822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae1481b55f64c638ec46a5d0ff7f51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } }
I20260806 01:05:51.677165 3822 leader_election.cc:302] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ae1481b55f64c638ec46a5d0ff7f51d; no voters:
I20260806 01:05:51.677646 3822 leader_election.cc:288] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:51.677762 3838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:51.678453 3838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [term 1 LEADER]: Becoming Leader. State: Replica: 4ae1481b55f64c638ec46a5d0ff7f51d, State: Running, Role: LEADER
I20260806 01:05:51.678992 3838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae1481b55f64c638ec46a5d0ff7f51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } }
I20260806 01:05:51.679595 3822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:51.681806 3841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ae1481b55f64c638ec46a5d0ff7f51d. Latest consensus state: current_term: 1 leader_uuid: "4ae1481b55f64c638ec46a5d0ff7f51d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae1481b55f64c638ec46a5d0ff7f51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } } }
I20260806 01:05:51.682281 3841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:51.681836 3839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ae1481b55f64c638ec46a5d0ff7f51d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ae1481b55f64c638ec46a5d0ff7f51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } } }
I20260806 01:05:51.682672 3839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:51.683938 3844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:51.689338 3844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:51.697479 3844 catalog_manager.cc:1347] Generated new cluster ID: 31718b10f7684e25a376942b919ae5ce
I20260806 01:05:51.697582 3844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:51.736653 3844 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:51.737100 3844 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:51.747792 3844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ae1481b55f64c638ec46a5d0ff7f51d: Generated new TSK 0
I20260806 01:05:51.748651 3844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:05:51.765647 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 3717
W20260806 01:05:51.766624 3814 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52343
W20260806 01:05:51.767539 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 3717: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:05:51.767704 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 3717
2026-08-06T01:05:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2089 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:51 dist-test-slave-x4qh krb5kdc[3867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:51 dist-test-slave-x4qh krb5kdc[3867](info): set up 2 sockets
Aug 06 01:05:51 dist-test-slave-x4qh krb5kdc[3867](info): commencing operation
krb5kdc: starting...
W20260806 01:05:53.026569 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.003s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:53 dist-test-slave-x4qh krb5kdc[3867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:05:53.298430 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:53.622383 3883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:53.622680 3883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:53.622841 3883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:53.629881 3883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:53.630023 3883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:53.630119 3883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:53.630223 3883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:53.630295 3883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:53.638749 3883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:53.639999 3883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:53.641237 3883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:53.652834 3891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:53.652889 3892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:53.652853 3894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:53.653863 3883 server_base.cc:1034] running on GCE node
I20260806 01:05:53.654547 3883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:53.655748 3883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:53.656983 3883 hybrid_clock.cc:648] HybridClock initialized: now 1785978353656942 us; error 44 us; skew 500 ppm
I20260806 01:05:53.657335 3883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:53 dist-test-slave-x4qh krb5kdc[3867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978353, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:53.660310 3883 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:05:53.661646 3883 webserver.cc:460] Webserver started at http://127.0.0.1:35583/ using document root <none> and password file <none>
I20260806 01:05:53.662282 3883 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:53.662448 3883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:53.662730 3883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:53.665542 3883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "7e7976f3540f40aa9178c5f5f3747f71"
format_stamp: "Formatted at 2026-08-06 01:05:53 on dist-test-slave-x4qh"
server_key: "6ceeb06957b0ff00d4f720a1aa2bcf8d"
server_key_iv: "d4d39cd6d58236294b92edb6facf736f"
server_key_version: "encryptionkey@0"
I20260806 01:05:53.666282 3883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "7e7976f3540f40aa9178c5f5f3747f71"
format_stamp: "Formatted at 2026-08-06 01:05:53 on dist-test-slave-x4qh"
server_key: "6ceeb06957b0ff00d4f720a1aa2bcf8d"
server_key_iv: "d4d39cd6d58236294b92edb6facf736f"
server_key_version: "encryptionkey@0"
I20260806 01:05:53.671022 3883 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 01:05:53.674139 3907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:53.675246 3883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:05:53.675473 3883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "7e7976f3540f40aa9178c5f5f3747f71"
format_stamp: "Formatted at 2026-08-06 01:05:53 on dist-test-slave-x4qh"
server_key: "6ceeb06957b0ff00d4f720a1aa2bcf8d"
server_key_iv: "d4d39cd6d58236294b92edb6facf736f"
server_key_version: "encryptionkey@0"
I20260806 01:05:53.675653 3883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:53.692055 3883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:53.696237 3883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:53.696527 3883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:53.714681 3883 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41823
I20260806 01:05:53.714720 3978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41823 every 8 connection(s)
I20260806 01:05:53.715952 3883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:53.719527 3981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:53.724377 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 3883
I20260806 01:05:53.724555 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:53.724958 5554 external_mini_cluster.cc:1428] Setting key 46c49a437d9ad52afedd0a8b8001e5a7
I20260806 01:05:53.730300 3981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71: Bootstrap starting.
I20260806 01:05:53.733711 3981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:53.734932 3981 log.cc:824] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:53.738145 3981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71: No bootstrap required, opened a new log
Aug 06 01:05:53 dist-test-slave-x4qh krb5kdc[3867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:05:53.746093 3981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7976f3540f40aa9178c5f5f3747f71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41823 } }
I20260806 01:05:53.746465 3981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:53.746618 3981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7976f3540f40aa9178c5f5f3747f71, State: Initialized, Role: FOLLOWER
I20260806 01:05:53.747198 3981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7976f3540f40aa9178c5f5f3747f71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41823 } }
I20260806 01:05:53.747483 3981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:53.747613 3981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:53.747798 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:53.726676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57703 (local address 127.0.0.1:41823)
0806 01:05:53.727531 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:53.727541 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:53.727578 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:05:53.729211 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:53.729232 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:53.729567 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:53.729862 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:53.731652 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:53.732911 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:53.734823 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:53.735694 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:53.735903 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:53.740545 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:53.740553 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:53.740570 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:53.740624 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:53.743720 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:53.744492 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:53.744501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:53.744507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:53.744678 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:53.745065 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:53.745071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:53.745074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:53.745598 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:53.745903 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:53.746352 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 01:05:53.746805 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":379,"threads_started":1}
I20260806 01:05:53.747802 3981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:53.750114 3981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7976f3540f40aa9178c5f5f3747f71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41823 } }
I20260806 01:05:53.750638 3981 leader_election.cc:302] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e7976f3540f40aa9178c5f5f3747f71; no voters:
I20260806 01:05:53.751273 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:05:53.751279 3981 leader_election.cc:288] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:53.751403 3988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:53.752743 3988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [term 1 LEADER]: Becoming Leader. State: Replica: 7e7976f3540f40aa9178c5f5f3747f71, State: Running, Role: LEADER
I20260806 01:05:53.753346 3988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7976f3540f40aa9178c5f5f3747f71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41823 } }
I20260806 01:05:53.753692 3981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:53.756161 3994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e7976f3540f40aa9178c5f5f3747f71. Latest consensus state: current_term: 1 leader_uuid: "7e7976f3540f40aa9178c5f5f3747f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7976f3540f40aa9178c5f5f3747f71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41823 } } }
I20260806 01:05:53.756595 3994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:53.757083 3993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e7976f3540f40aa9178c5f5f3747f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7976f3540f40aa9178c5f5f3747f71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41823 } } }
I20260806 01:05:53.757426 3993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:53.757580 4006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:53.771611 4006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:53.789582 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:53.757906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60331 (local address 127.0.0.1:41823)
0806 01:05:53.758063 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:53.758068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:53.768146 (+ 10078us) server_negotiation.cc:407] Connection header received
0806 01:05:53.768737 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:53.768745 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:53.768840 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:53.768939 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:53.771413 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:53.772873 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:53.775251 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:53.776254 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:53.776413 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:53.780417 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:53.780427 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:53.780436 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:05:53.780490 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:53.785150 (+ 4660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:53.786216 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:53.786224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:53.786230 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:53.786431 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:05:53.786980 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:05:53.786988 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:05:53.786993 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:05:53.787613 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:53.787925 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:53.788707 (+ 782us) server_negotiation.cc:299] Negotiation successful
0806 01:05:53.789033 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:05:53.789932 4006 catalog_manager.cc:1347] Generated new cluster ID: 63f1b11000594c1285cec9f16713d678
I20260806 01:05:53.790184 4006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:53.816112 4006 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:53.817668 4006 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:53.826571 4006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e7976f3540f40aa9178c5f5f3747f71: Generated new TSK 0
I20260806 01:05:53.827310 4006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:05:53.835453 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 3883
I20260806 01:05:53.836731 3971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:05:53.925410 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 3883
2026-08-06T01:05:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2158 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:54 dist-test-slave-x4qh krb5kdc[4030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:54 dist-test-slave-x4qh krb5kdc[4030](info): set up 2 sockets
Aug 06 01:05:54 dist-test-slave-x4qh krb5kdc[4030](info): commencing operation
krb5kdc: starting...
W20260806 01:05:55.196115 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:55 dist-test-slave-x4qh krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:05:55.481709 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39069
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:55.812016 4049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:55.812289 4049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:55.812491 4049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:55.818764 4049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:55.818856 4049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:55.818902 4049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:55.818997 4049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:55.819056 4049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:55.828279 4049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:55.829383 4049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:55.830724 4049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:55.842675 4062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:55.842792 4060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:55.842823 4058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:55.843506 4049 server_base.cc:1034] running on GCE node
I20260806 01:05:55.844246 4049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:55.845564 4049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:55.846796 4049 hybrid_clock.cc:648] HybridClock initialized: now 1785978355846755 us; error 37 us; skew 500 ppm
I20260806 01:05:55.847128 4049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:55 dist-test-slave-x4qh krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978355, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:55.850193 4049 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:05:55.851684 4049 webserver.cc:460] Webserver started at http://127.0.0.1:42697/ using document root <none> and password file <none>
I20260806 01:05:55.852280 4049 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:55.852380 4049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:55.852619 4049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:55.855356 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "a8cb07460abc41b3963c13cbaea02c43"
format_stamp: "Formatted at 2026-08-06 01:05:55 on dist-test-slave-x4qh"
server_key: "8d0a073396790b586a3cfc9895cac4ab"
server_key_iv: "a5f6d70504f81485205b8419713a4e80"
server_key_version: "encryptionkey@0"
I20260806 01:05:55.856079 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "a8cb07460abc41b3963c13cbaea02c43"
format_stamp: "Formatted at 2026-08-06 01:05:55 on dist-test-slave-x4qh"
server_key: "8d0a073396790b586a3cfc9895cac4ab"
server_key_iv: "a5f6d70504f81485205b8419713a4e80"
server_key_version: "encryptionkey@0"
I20260806 01:05:55.860841 4049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 01:05:55.864054 4071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:55.865255 4049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:05:55.865394 4049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "a8cb07460abc41b3963c13cbaea02c43"
format_stamp: "Formatted at 2026-08-06 01:05:55 on dist-test-slave-x4qh"
server_key: "8d0a073396790b586a3cfc9895cac4ab"
server_key_iv: "a5f6d70504f81485205b8419713a4e80"
server_key_version: "encryptionkey@0"
I20260806 01:05:55.865557 4049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:55.894949 4049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:55.898689 4049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:55.898953 4049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:55.918398 4049 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39069
I20260806 01:05:55.918434 4141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39069 every 8 connection(s)
I20260806 01:05:55.919755 4049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:55.923205 4142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:55.924419 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 4049
I20260806 01:05:55.924592 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:55.925122 5554 external_mini_cluster.cc:1428] Setting key a7202d19bc5321724016d6b2bfe0ee81
I20260806 01:05:55.937142 4142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43: Bootstrap starting.
I20260806 01:05:55.939091 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:55.927012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46997 (local address 127.0.0.1:39069)
0806 01:05:55.927790 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:55.927804 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:55.927856 (+ 52us) server_negotiation.cc:407] Connection header received
0806 01:05:55.929665 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:55.929691 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:55.930176 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:55.930610 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:55.931087 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:55.932916 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:55.934162 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:55.935184 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:55.935356 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:55.936415 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:55.936425 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:55.936441 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:55.937113 (+ 672us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:55.937216 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:55.937250 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:55.937368 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:55.937635 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 01:05:55.937879 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":349,"threads_started":1}
I20260806 01:05:55.940946 4142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:55.942152 4142 log.cc:824] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:55.942626 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:05:55.944691 4142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43: No bootstrap required, opened a new log
I20260806 01:05:55.953136 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:55.946964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35685 (local address 127.0.0.1:39069)
0806 01:05:55.947131 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:55.947138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:55.947530 (+ 392us) server_negotiation.cc:407] Connection header received
0806 01:05:55.947667 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:55.947673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:55.947753 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:55.947854 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:55.948256 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:55.949365 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:55.950661 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:05:55.951342 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:05:55.951480 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:05:55.952441 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:55.952447 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:55.952451 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:55.952468 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:55.952512 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:55.952530 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:55.952610 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:55.952765 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 01:05:55.952842 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:05:55.953279 4142 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8cb07460abc41b3963c13cbaea02c43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39069 } }
I20260806 01:05:55.953673 4142 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:55.953787 4142 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8cb07460abc41b3963c13cbaea02c43, State: Initialized, Role: FOLLOWER
I20260806 01:05:55.954396 4142 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8cb07460abc41b3963c13cbaea02c43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39069 } }
I20260806 01:05:55.954603 4142 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:55.954696 4142 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:55.954815 4142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:55.956736 4142 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8cb07460abc41b3963c13cbaea02c43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39069 } }
I20260806 01:05:55.957113 4142 leader_election.cc:302] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8cb07460abc41b3963c13cbaea02c43; no voters:
I20260806 01:05:55.957598 4142 leader_election.cc:288] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:55.957744 4156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:55.958423 4156 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [term 1 LEADER]: Becoming Leader. State: Replica: a8cb07460abc41b3963c13cbaea02c43, State: Running, Role: LEADER
I20260806 01:05:55.958952 4156 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8cb07460abc41b3963c13cbaea02c43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39069 } }
I20260806 01:05:55.959645 4142 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:55.961956 4159 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8cb07460abc41b3963c13cbaea02c43. Latest consensus state: current_term: 1 leader_uuid: "a8cb07460abc41b3963c13cbaea02c43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8cb07460abc41b3963c13cbaea02c43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39069 } } }
I20260806 01:05:55.961931 4157 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8cb07460abc41b3963c13cbaea02c43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8cb07460abc41b3963c13cbaea02c43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39069 } } }
I20260806 01:05:55.962386 4159 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:55.962386 4157 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:55.965368 4163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:55.971248 4163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:55.981030 4163 catalog_manager.cc:1347] Generated new cluster ID: 606d6b644658478b86e3b724888023c5
I20260806 01:05:55.981144 4163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:55.992931 4163 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:55.994110 4163 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:55.999550 4163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8cb07460abc41b3963c13cbaea02c43: Generated new TSK 0
I20260806 01:05:56.000279 4163 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:05:56.009334 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4049
W20260806 01:05:56.010325 4137 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46997
W20260806 01:05:56.011094 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4049: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:05:56.011284 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 4049
2026-08-06T01:05:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2085 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:56 dist-test-slave-x4qh krb5kdc[4183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:56 dist-test-slave-x4qh krb5kdc[4183](info): set up 2 sockets
Aug 06 01:05:56 dist-test-slave-x4qh krb5kdc[4183](info): commencing operation
krb5kdc: starting...
W20260806 01:05:57.266115 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:57 dist-test-slave-x4qh krb5kdc[4183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:05:57.544351 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:57.886715 4200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:57.887005 4200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:57.887156 4200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:57.894129 4200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:57.894223 4200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:57.894337 4200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:57.894441 4200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:57.894520 4200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:57.904294 4200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:57.905580 4200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:57.906895 4200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:57.918708 4206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:57.918736 4207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:57.918641 4210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:05:57.919612 4200 server_base.cc:1034] running on GCE node
I20260806 01:05:57.920430 4200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:05:57.921756 4200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:05:57.922981 4200 hybrid_clock.cc:648] HybridClock initialized: now 1785978357922931 us; error 38 us; skew 500 ppm
I20260806 01:05:57.923302 4200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:05:57 dist-test-slave-x4qh krb5kdc[4183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978357, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:05:57.926519 4200 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:05:57.927989 4200 webserver.cc:460] Webserver started at http://127.0.0.1:34141/ using document root <none> and password file <none>
I20260806 01:05:57.928640 4200 fs_manager.cc:359] Metadata directory not provided
I20260806 01:05:57.928789 4200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:05:57.929090 4200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:05:57.932231 4200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "fc44454cf46644c495883398c5e005c6"
format_stamp: "Formatted at 2026-08-06 01:05:57 on dist-test-slave-x4qh"
server_key: "ff59fb480d4e313744fb9249ea327968"
server_key_iv: "0d7f6eee0cd2ca18684a64ff35543bfb"
server_key_version: "encryptionkey@0"
I20260806 01:05:57.933028 4200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "fc44454cf46644c495883398c5e005c6"
format_stamp: "Formatted at 2026-08-06 01:05:57 on dist-test-slave-x4qh"
server_key: "ff59fb480d4e313744fb9249ea327968"
server_key_iv: "0d7f6eee0cd2ca18684a64ff35543bfb"
server_key_version: "encryptionkey@0"
I20260806 01:05:57.938036 4200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:05:57.941336 4219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:05:57.942531 4200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:05:57.942708 4200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "fc44454cf46644c495883398c5e005c6"
format_stamp: "Formatted at 2026-08-06 01:05:57 on dist-test-slave-x4qh"
server_key: "ff59fb480d4e313744fb9249ea327968"
server_key_iv: "0d7f6eee0cd2ca18684a64ff35543bfb"
server_key_version: "encryptionkey@0"
I20260806 01:05:57.942914 4200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:05:57.948400 4200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:05:57.952111 4200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:05:57.952396 4200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:05:57.970153 4200 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41717
I20260806 01:05:57.970299 4300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41717 every 8 connection(s)
I20260806 01:05:57.971544 4200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:05:57.975338 4302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:05:57.980768 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 4200
I20260806 01:05:57.980979 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:05:57.981528 5554 external_mini_cluster.cc:1428] Setting key d573d16227641b1d6ed1b863c0185342
I20260806 01:05:57.987131 4302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6: Bootstrap starting.
I20260806 01:05:57.989284 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:57.983329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33169 (local address 127.0.0.1:41717)
0806 01:05:57.984203 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:57.984214 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:57.984255 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:05:57.985962 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:57.985982 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:57.986285 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:57.986580 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:57.987755 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:57.987761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:57.987774 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:57.987796 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:57.987851 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:57.987870 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:57.988034 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:57.988192 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 01:05:57.988363 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":404,"threads_started":1}
I20260806 01:05:57.990674 4302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6: Neither blocks nor log segments found. Creating new log.
I20260806 01:05:57.992012 4302 log.cc:824] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6: Log is configured to *not* fsync() on all Append() calls
I20260806 01:05:57.992833 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:05:57.995090 4302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6: No bootstrap required, opened a new log
I20260806 01:05:58.000563 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:05:57.997944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45987 (local address 127.0.0.1:41717)
0806 01:05:57.998114 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:05:57.998122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:05:57.998596 (+ 474us) server_negotiation.cc:407] Connection header received
0806 01:05:57.998777 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:05:57.998780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:05:57.998820 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:05:57.998897 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:05:57.999927 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:05:57.999930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:05:57.999933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:05:57.999947 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:05:57.999984 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:05:57.999997 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:05:58.000085 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 01:05:58.000244 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 01:05:58.000287 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 01:05:58.004786 4302 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44454cf46644c495883398c5e005c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41717 } }
I20260806 01:05:58.005156 4302 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:05:58.005285 4302 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc44454cf46644c495883398c5e005c6, State: Initialized, Role: FOLLOWER
I20260806 01:05:58.005853 4302 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44454cf46644c495883398c5e005c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41717 } }
I20260806 01:05:58.006145 4302 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:05:58.006304 4302 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:05:58.006498 4302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:05:58.008924 4302 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44454cf46644c495883398c5e005c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41717 } }
I20260806 01:05:58.009418 4302 leader_election.cc:302] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc44454cf46644c495883398c5e005c6; no voters:
I20260806 01:05:58.010016 4302 leader_election.cc:288] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:05:58.010121 4316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:05:58.010828 4316 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [term 1 LEADER]: Becoming Leader. State: Replica: fc44454cf46644c495883398c5e005c6, State: Running, Role: LEADER
I20260806 01:05:58.011449 4316 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44454cf46644c495883398c5e005c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41717 } }
I20260806 01:05:58.012259 4302 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:05:58.014026 4317 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc44454cf46644c495883398c5e005c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44454cf46644c495883398c5e005c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41717 } } }
I20260806 01:05:58.014472 4317 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:58.014457 4318 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc44454cf46644c495883398c5e005c6. Latest consensus state: current_term: 1 leader_uuid: "fc44454cf46644c495883398c5e005c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44454cf46644c495883398c5e005c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41717 } } }
I20260806 01:05:58.014775 4318 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6 [sys.catalog]: This master's current role is: LEADER
I20260806 01:05:58.024323 4326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:05:58.029431 4326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:05:58.037878 4326 catalog_manager.cc:1347] Generated new cluster ID: b93832892b054c97a73e0c0573653055
I20260806 01:05:58.037989 4326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:05:58.055644 4326 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:05:58.056105 4326 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:05:58.068914 4326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc44454cf46644c495883398c5e005c6: Generated new TSK 0
I20260806 01:05:58.069831 4326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:05:58.081676 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4200
W20260806 01:05:58.082860 4295 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33169
W20260806 01:05:58.083652 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4200: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:05:58.083806 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 4200
2026-08-06T01:05:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2069 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:05:58 dist-test-slave-x4qh krb5kdc[4345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:05:58 dist-test-slave-x4qh krb5kdc[4345](info): set up 2 sockets
Aug 06 01:05:58 dist-test-slave-x4qh krb5kdc[4345](info): commencing operation
krb5kdc: starting...
W20260806 01:05:59.342386 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:05:59 dist-test-slave-x4qh krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:05:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:05:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:05:59.638213 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:05:59.968806 4363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:05:59.969117 4363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:05:59.969285 4363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:05:59.976275 4363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:05:59.976414 4363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:05:59.976507 4363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:05:59.976610 4363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:05:59.976696 4363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:05:59.986047 4363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:05:59.987208 4363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:05:59.988467 4363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:05:59.999807 4369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:59.999760 4371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:05:59.999837 4368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:00.000759 4363 server_base.cc:1034] running on GCE node
I20260806 01:06:00.001497 4363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:00.002820 4363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:00.004041 4363 hybrid_clock.cc:648] HybridClock initialized: now 1785978360003999 us; error 50 us; skew 500 ppm
I20260806 01:06:00.004374 4363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:00 dist-test-slave-x4qh krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978360, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:00.007472 4363 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:06:00.008786 4363 webserver.cc:460] Webserver started at http://127.0.0.1:35675/ using document root <none> and password file <none>
I20260806 01:06:00.009344 4363 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:00.009434 4363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:00.009657 4363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:00.012668 4363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "658c7424d82649a4b5af3ede3d0c6b85"
format_stamp: "Formatted at 2026-08-06 01:06:00 on dist-test-slave-x4qh"
server_key: "ac3b08306b895cc90798b24c3e7db0b7"
server_key_iv: "ac6dfe7372a7ae1980a898ed3b8ca11c"
server_key_version: "encryptionkey@0"
I20260806 01:06:00.013391 4363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "658c7424d82649a4b5af3ede3d0c6b85"
format_stamp: "Formatted at 2026-08-06 01:06:00 on dist-test-slave-x4qh"
server_key: "ac3b08306b895cc90798b24c3e7db0b7"
server_key_iv: "ac6dfe7372a7ae1980a898ed3b8ca11c"
server_key_version: "encryptionkey@0"
I20260806 01:06:00.018247 4363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:06:00.021526 4384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:00.022730 4363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:06:00.022857 4363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "658c7424d82649a4b5af3ede3d0c6b85"
format_stamp: "Formatted at 2026-08-06 01:06:00 on dist-test-slave-x4qh"
server_key: "ac3b08306b895cc90798b24c3e7db0b7"
server_key_iv: "ac6dfe7372a7ae1980a898ed3b8ca11c"
server_key_version: "encryptionkey@0"
I20260806 01:06:00.022998 4363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:00.045120 4363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:00.048887 4363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:00.049103 4363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:00.067260 4363 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38911
I20260806 01:06:00.067337 4453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38911 every 8 connection(s)
I20260806 01:06:00.068536 4363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:00.072055 4454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:00.076813 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 4363
I20260806 01:06:00.077037 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:00.077692 5554 external_mini_cluster.cc:1428] Setting key 8611221a41a376e32db2986614579a9d
I20260806 01:06:00.084429 4454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85: Bootstrap starting.
I20260806 01:06:00.088119 4454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:00.089308 4454 log.cc:824] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:00.092679 4454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85: No bootstrap required, opened a new log
Aug 06 01:06:00 dist-test-slave-x4qh krb5kdc[4345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:06:00.100356 4454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c7424d82649a4b5af3ede3d0c6b85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } }
I20260806 01:06:00.100888 4454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:00.101028 4454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658c7424d82649a4b5af3ede3d0c6b85, State: Initialized, Role: FOLLOWER
I20260806 01:06:00.101784 4458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:00.079632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:34359 (local address 127.0.0.1:38911)
0806 01:06:00.080377 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:00.080387 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:00.080428 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:06:00.082630 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:00.082652 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:00.082985 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:00.083266 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:00.085449 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:00.086753 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:00.088676 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:00.089312 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:00.089474 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:00.094509 (+ 5035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:00.094528 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:00.094540 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:00.094579 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:00.097555 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:00.098291 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:00.098297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:00.098302 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:00.098466 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:00.098973 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:00.098976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:00.098978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:00.099499 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:00.099848 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:00.100307 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 01:06:00.100609 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":375,"threads_started":1}
I20260806 01:06:00.101634 4454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c7424d82649a4b5af3ede3d0c6b85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } }
I20260806 01:06:00.101985 4454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:00.102171 4454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:00.102380 4454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:00.105181 4454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c7424d82649a4b5af3ede3d0c6b85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } }
I20260806 01:06:00.105547 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:00.105736 4454 leader_election.cc:302] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 658c7424d82649a4b5af3ede3d0c6b85; no voters:
I20260806 01:06:00.106403 4454 leader_election.cc:288] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:00.106525 4460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:00.108920 4454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:00.115777 4460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [term 1 LEADER]: Becoming Leader. State: Replica: 658c7424d82649a4b5af3ede3d0c6b85, State: Running, Role: LEADER
I20260806 01:06:00.116519 4460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c7424d82649a4b5af3ede3d0c6b85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } }
I20260806 01:06:00.120000 4465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 658c7424d82649a4b5af3ede3d0c6b85. Latest consensus state: current_term: 1 leader_uuid: "658c7424d82649a4b5af3ede3d0c6b85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c7424d82649a4b5af3ede3d0c6b85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } } }
I20260806 01:06:00.120558 4465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:00.121165 4463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "658c7424d82649a4b5af3ede3d0c6b85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c7424d82649a4b5af3ede3d0c6b85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38911 } } }
I20260806 01:06:00.121407 4463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:00.126329 4485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:00.132001 4485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:00.135840 4458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:00.116829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:58183 (local address 127.0.0.1:38911)
0806 01:06:00.117020 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:00.117029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:00.117511 (+ 482us) server_negotiation.cc:407] Connection header received
0806 01:06:00.117815 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:00.117822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:00.117894 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:00.117992 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:00.120152 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:00.121326 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:00.123523 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:00.124184 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:00.124376 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:00.126901 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:00.126909 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:00.126916 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:00.126955 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:00.130622 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:00.131332 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:00.131337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:00.131341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:00.131499 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:00.131936 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:00.131952 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:00.131955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:00.132478 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:00.134077 (+ 1599us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:00.134319 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 01:06:00.134515 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 01:06:00.141101 4485 catalog_manager.cc:1347] Generated new cluster ID: 5eda7ad00b114323b86850f81ff7a86f
I20260806 01:06:00.141227 4485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:00.152647 4485 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:00.153839 4485 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:00.159643 4485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 658c7424d82649a4b5af3ede3d0c6b85: Generated new TSK 0
I20260806 01:06:00.160514 4485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:00.172442 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4363
I20260806 01:06:00.173481 4449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:00.260099 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 4363
W20260806 01:06:00.281795 4342 connection.cc:535] client connection to 127.0.0.1:38911 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38911): Connection reset by peer (error 104)
2026-08-06T01:06:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2176 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:00 dist-test-slave-x4qh krb5kdc[4505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:00 dist-test-slave-x4qh krb5kdc[4505](info): set up 2 sockets
Aug 06 01:06:00 dist-test-slave-x4qh krb5kdc[4505](info): commencing operation
krb5kdc: starting...
W20260806 01:06:01.501837 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:01 dist-test-slave-x4qh krb5kdc[4505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:06:01.792495 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:02.121625 4525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:02.121881 4525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:02.122026 4525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:02.128810 4525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:02.128909 4525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:02.128996 4525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:02.129107 4525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:02.129184 4525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:02.138265 4525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:02.139436 4525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:02.140746 4525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:02.151815 4532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:02.151944 4533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:02.152285 4537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:02.152928 4525 server_base.cc:1034] running on GCE node
I20260806 01:06:02.153692 4525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:02.154980 4525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:02.156181 4525 hybrid_clock.cc:648] HybridClock initialized: now 1785978362156143 us; error 33 us; skew 500 ppm
I20260806 01:06:02.156530 4525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:02 dist-test-slave-x4qh krb5kdc[4505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978362, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:02.159612 4525 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:06:02.160992 4525 webserver.cc:460] Webserver started at http://127.0.0.1:41017/ using document root <none> and password file <none>
I20260806 01:06:02.161639 4525 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:02.161801 4525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:02.162101 4525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:02.164911 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "e93566bbd74b4934b26ff5b636501815"
format_stamp: "Formatted at 2026-08-06 01:06:02 on dist-test-slave-x4qh"
server_key: "d2d5fe8250cd983830894d9cc9343112"
server_key_iv: "addffe7b0e3aa38ac7c005260acc07b1"
server_key_version: "encryptionkey@0"
I20260806 01:06:02.165706 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "e93566bbd74b4934b26ff5b636501815"
format_stamp: "Formatted at 2026-08-06 01:06:02 on dist-test-slave-x4qh"
server_key: "d2d5fe8250cd983830894d9cc9343112"
server_key_iv: "addffe7b0e3aa38ac7c005260acc07b1"
server_key_version: "encryptionkey@0"
I20260806 01:06:02.170603 4525 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:06:02.173900 4551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:02.175004 4525 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:06:02.175194 4525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "e93566bbd74b4934b26ff5b636501815"
format_stamp: "Formatted at 2026-08-06 01:06:02 on dist-test-slave-x4qh"
server_key: "d2d5fe8250cd983830894d9cc9343112"
server_key_iv: "addffe7b0e3aa38ac7c005260acc07b1"
server_key_version: "encryptionkey@0"
I20260806 01:06:02.175428 4525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:02.193068 4525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:02.197489 4525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:02.197786 4525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:02.216185 4525 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37209
I20260806 01:06:02.216203 4617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37209 every 8 connection(s)
I20260806 01:06:02.217643 4525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:02.221278 4618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:02.227085 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 4525
I20260806 01:06:02.227245 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:02.227730 5554 external_mini_cluster.cc:1428] Setting key f8ffd4a87ae7b2121aa367b6e31e1b38
I20260806 01:06:02.231132 4618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815: Bootstrap starting.
I20260806 01:06:02.234311 4618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:02.235469 4618 log.cc:824] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:02.238268 4618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815: No bootstrap required, opened a new log
I20260806 01:06:02.241328 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:02.229521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:39823 (local address 127.0.0.1:37209)
0806 01:06:02.230304 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:02.230314 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:02.230356 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:06:02.232248 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:02.232269 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:02.232633 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:02.232977 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:02.233544 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:02.234818 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:02.236510 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:02.237120 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:02.237252 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:02.238800 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:02.238812 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:02.238830 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:02.239528 (+ 698us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:02.239600 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:02.239624 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:02.239800 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:02.240120 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 01:06:02.240285 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":348,"threads_started":1}
I20260806 01:06:02.245105 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:02.246872 4618 raft_consensus.cc:348] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93566bbd74b4934b26ff5b636501815" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37209 } }
I20260806 01:06:02.247216 4618 raft_consensus.cc:374] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:02.247336 4618 raft_consensus.cc:729] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e93566bbd74b4934b26ff5b636501815, State: Initialized, Role: FOLLOWER
I20260806 01:06:02.247963 4618 consensus_queue.cc:260] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93566bbd74b4934b26ff5b636501815" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37209 } }
I20260806 01:06:02.248209 4618 raft_consensus.cc:388] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:02.248383 4618 raft_consensus.cc:482] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:02.248562 4618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:02.251402 4618 raft_consensus.cc:504] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93566bbd74b4934b26ff5b636501815" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37209 } }
I20260806 01:06:02.251962 4618 leader_election.cc:302] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e93566bbd74b4934b26ff5b636501815; no voters:
I20260806 01:06:02.252609 4618 leader_election.cc:288] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:02.252704 4638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:02.253875 4638 raft_consensus.cc:686] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [term 1 LEADER]: Becoming Leader. State: Replica: e93566bbd74b4934b26ff5b636501815, State: Running, Role: LEADER
I20260806 01:06:02.254400 4638 consensus_queue.cc:237] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93566bbd74b4934b26ff5b636501815" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37209 } }
I20260806 01:06:02.254582 4618 sys_catalog.cc:564] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:02.257135 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:02.249346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:56047 (local address 127.0.0.1:37209)
0806 01:06:02.249558 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:02.249564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:02.249897 (+ 333us) server_negotiation.cc:407] Connection header received
0806 01:06:02.250028 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:02.250031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:02.250088 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:02.250184 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:02.250827 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:02.251668 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:02.253538 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:02.254199 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:02.254305 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:02.255783 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:02.255790 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:02.255798 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:02.255825 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:02.255895 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:02.255923 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:02.256068 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:02.256572 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 01:06:02.256635 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:06:02.258716 4639 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e93566bbd74b4934b26ff5b636501815" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93566bbd74b4934b26ff5b636501815" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37209 } } }
I20260806 01:06:02.259068 4639 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:02.259547 4640 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e93566bbd74b4934b26ff5b636501815. Latest consensus state: current_term: 1 leader_uuid: "e93566bbd74b4934b26ff5b636501815" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93566bbd74b4934b26ff5b636501815" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37209 } } }
I20260806 01:06:02.259848 4640 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:02.261320 4651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:02.266170 4651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:02.273689 4651 catalog_manager.cc:1347] Generated new cluster ID: ce4c255290484fc9ae54cd4adb8411b6
I20260806 01:06:02.273788 4651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:02.283806 4651 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:02.285478 4651 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:02.292625 4651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e93566bbd74b4934b26ff5b636501815: Generated new TSK 0
I20260806 01:06:02.293516 4651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:02.302363 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4525
W20260806 01:06:02.303731 4612 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.93:39823
W20260806 01:06:02.304528 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4525: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:02.304711 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 4525
W20260806 01:06:02.328207 4498 connection.cc:535] client connection to 127.0.0.1:37209 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37209): Connection reset by peer (error 104)
2026-08-06T01:06:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2046 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:02 dist-test-slave-x4qh krb5kdc[4663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:02 dist-test-slave-x4qh krb5kdc[4663](info): set up 2 sockets
Aug 06 01:06:02 dist-test-slave-x4qh krb5kdc[4663](info): commencing operation
krb5kdc: starting...
W20260806 01:06:03.571765 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:03 dist-test-slave-x4qh krb5kdc[4663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:06:03.884531 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:04.217864 4680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:04.218123 4680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:04.218241 4680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:04.225163 4680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:04.225310 4680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:04.225417 4680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:04.225519 4680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:04.225616 4680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:04.234894 4680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:04.236097 4680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:04.237279 4680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:04.248891 4685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:04.248848 4686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:04.248936 4688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:04.252832 4680 server_base.cc:1034] running on GCE node
I20260806 01:06:04.253655 4680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:04.255002 4680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:04.256217 4680 hybrid_clock.cc:648] HybridClock initialized: now 1785978364256172 us; error 63 us; skew 500 ppm
I20260806 01:06:04.256556 4680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:04 dist-test-slave-x4qh krb5kdc[4663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978364, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:04.264201 4680 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:06:04.265746 4680 webserver.cc:460] Webserver started at http://127.0.0.1:44497/ using document root <none> and password file <none>
I20260806 01:06:04.266464 4680 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:04.266621 4680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:04.266922 4680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:04.269806 4680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "0537a4b46cb94a2187c387ec3b2d6c77"
format_stamp: "Formatted at 2026-08-06 01:06:04 on dist-test-slave-x4qh"
server_key: "18b1ce68a46acf051c39c3f0e78c6f91"
server_key_iv: "b23bb7c704f441a918af68047579616c"
server_key_version: "encryptionkey@0"
I20260806 01:06:04.270599 4680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "0537a4b46cb94a2187c387ec3b2d6c77"
format_stamp: "Formatted at 2026-08-06 01:06:04 on dist-test-slave-x4qh"
server_key: "18b1ce68a46acf051c39c3f0e78c6f91"
server_key_iv: "b23bb7c704f441a918af68047579616c"
server_key_version: "encryptionkey@0"
I20260806 01:06:04.275821 4680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:06:04.279042 4696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:04.280313 4680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:06:04.280515 4680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "0537a4b46cb94a2187c387ec3b2d6c77"
format_stamp: "Formatted at 2026-08-06 01:06:04 on dist-test-slave-x4qh"
server_key: "18b1ce68a46acf051c39c3f0e78c6f91"
server_key_iv: "b23bb7c704f441a918af68047579616c"
server_key_version: "encryptionkey@0"
I20260806 01:06:04.280718 4680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:04.286338 4680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:04.290005 4680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:04.290318 4680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:04.309090 4680 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39701
I20260806 01:06:04.309168 4778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39701 every 8 connection(s)
I20260806 01:06:04.310420 4680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:04.313910 4782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:04.314386 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 4680
I20260806 01:06:04.314565 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:04.314976 5554 external_mini_cluster.cc:1428] Setting key 329be4428e40e52f3613e9dacda645bb
I20260806 01:06:04.322695 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:04.316557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:48879 (local address 127.0.0.1:39701)
0806 01:06:04.317349 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:04.317359 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:04.317396 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:06:04.319506 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:04.319527 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:04.319829 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:04.320139 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:04.321271 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:04.321277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:04.321289 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:04.321303 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:04.321365 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:04.321383 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:04.321551 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:04.321698 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 01:06:04.321868 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":371,"threads_started":1}
I20260806 01:06:04.326066 4782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77: Bootstrap starting.
I20260806 01:06:04.326125 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:04.329697 4782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:04.331036 4782 log.cc:824] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:04.333636 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:04.331215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:34795 (local address 127.0.0.1:39701)
0806 01:06:04.331397 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:04.331403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:04.331924 (+ 521us) server_negotiation.cc:407] Connection header received
0806 01:06:04.332090 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:04.332098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:04.332147 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:04.332242 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:04.333070 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:04.333074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:04.333079 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:04.333094 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:04.333134 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:04.333147 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:04.333256 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:04.333340 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 01:06:04.333393 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:06:04.333907 4782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77: No bootstrap required, opened a new log
I20260806 01:06:04.341111 4782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0537a4b46cb94a2187c387ec3b2d6c77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39701 } }
I20260806 01:06:04.341480 4782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:04.341599 4782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0537a4b46cb94a2187c387ec3b2d6c77, State: Initialized, Role: FOLLOWER
I20260806 01:06:04.342183 4782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0537a4b46cb94a2187c387ec3b2d6c77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39701 } }
I20260806 01:06:04.342442 4782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:04.342577 4782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:04.342756 4782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:04.345188 4782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0537a4b46cb94a2187c387ec3b2d6c77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39701 } }
I20260806 01:06:04.345659 4782 leader_election.cc:302] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0537a4b46cb94a2187c387ec3b2d6c77; no voters:
I20260806 01:06:04.346202 4782 leader_election.cc:288] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:04.346307 4794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:04.347015 4794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [term 1 LEADER]: Becoming Leader. State: Replica: 0537a4b46cb94a2187c387ec3b2d6c77, State: Running, Role: LEADER
I20260806 01:06:04.347615 4794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0537a4b46cb94a2187c387ec3b2d6c77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39701 } }
I20260806 01:06:04.348327 4782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:04.350365 4795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0537a4b46cb94a2187c387ec3b2d6c77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0537a4b46cb94a2187c387ec3b2d6c77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39701 } } }
I20260806 01:06:04.350829 4795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:04.351920 4796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0537a4b46cb94a2187c387ec3b2d6c77. Latest consensus state: current_term: 1 leader_uuid: "0537a4b46cb94a2187c387ec3b2d6c77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0537a4b46cb94a2187c387ec3b2d6c77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39701 } } }
I20260806 01:06:04.352182 4796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:04.353546 4807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:04.357983 4807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:04.365854 4807 catalog_manager.cc:1347] Generated new cluster ID: 74b7886755ac4437996506b3ecbfba0d
I20260806 01:06:04.366025 4807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:04.379070 4807 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:04.379544 4807 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:04.395071 4807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0537a4b46cb94a2187c387ec3b2d6c77: Generated new TSK 0
I20260806 01:06:04.396097 4807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:04.413789 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4680
W20260806 01:06:04.415427 4770 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.93:48879
W20260806 01:06:04.416255 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4680: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:04.416425 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 4680
2026-08-06T01:06:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2117 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:04 dist-test-slave-x4qh krb5kdc[4823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:04 dist-test-slave-x4qh krb5kdc[4823](info): set up 2 sockets
Aug 06 01:06:04 dist-test-slave-x4qh krb5kdc[4823](info): commencing operation
krb5kdc: starting...
W20260806 01:06:05.704799 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:05 dist-test-slave-x4qh krb5kdc[4823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:06:05.991868 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41895
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:06.313110 4843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:06.313374 4843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:06.313525 4843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:06.320413 4843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:06.320528 4843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:06.320642 4843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:06.320746 4843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:06.320827 4843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:06.330780 4843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41895
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:06.332165 4843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:06.333360 4843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:06.344774 4850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:06.344803 4854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:06.344930 4852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:06.345471 4843 server_base.cc:1034] running on GCE node
I20260806 01:06:06.346148 4843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:06.347479 4843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:06.348712 4843 hybrid_clock.cc:648] HybridClock initialized: now 1785978366348667 us; error 48 us; skew 500 ppm
I20260806 01:06:06.349119 4843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:06 dist-test-slave-x4qh krb5kdc[4823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:06.352188 4843 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:06:06.353475 4843 webserver.cc:460] Webserver started at http://127.0.0.1:39521/ using document root <none> and password file <none>
I20260806 01:06:06.354101 4843 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:06.354233 4843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:06.354502 4843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:06.357261 4843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "4f280bffa41a4b7f8958cf86205b6192"
format_stamp: "Formatted at 2026-08-06 01:06:06 on dist-test-slave-x4qh"
server_key: "b92ff2b8df7a41cc0f6dfbac9c3267eb"
server_key_iv: "dbd8118856091a53c054efaa01010fe6"
server_key_version: "encryptionkey@0"
I20260806 01:06:06.357986 4843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "4f280bffa41a4b7f8958cf86205b6192"
format_stamp: "Formatted at 2026-08-06 01:06:06 on dist-test-slave-x4qh"
server_key: "b92ff2b8df7a41cc0f6dfbac9c3267eb"
server_key_iv: "dbd8118856091a53c054efaa01010fe6"
server_key_version: "encryptionkey@0"
I20260806 01:06:06.363016 4843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 01:06:06.366182 4864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:06.367513 4843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:06:06.367655 4843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "4f280bffa41a4b7f8958cf86205b6192"
format_stamp: "Formatted at 2026-08-06 01:06:06 on dist-test-slave-x4qh"
server_key: "b92ff2b8df7a41cc0f6dfbac9c3267eb"
server_key_iv: "dbd8118856091a53c054efaa01010fe6"
server_key_version: "encryptionkey@0"
I20260806 01:06:06.367805 4843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:06.386514 4843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:06.389907 4843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:06.390161 4843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:06.408231 4843 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41895
I20260806 01:06:06.408267 4928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41895 every 8 connection(s)
I20260806 01:06:06.409436 4843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:06.412806 4930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:06.417900 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 4843
I20260806 01:06:06.418078 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:06.418622 5554 external_mini_cluster.cc:1428] Setting key 9305d892f5506be62547d186b6184dc1
I20260806 01:06:06.423350 4930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192: Bootstrap starting.
I20260806 01:06:06.426967 4930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:06.428143 4930 log.cc:824] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:06.431277 4930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192: No bootstrap required, opened a new log
Aug 06 01:06:06 dist-test-slave-x4qh krb5kdc[4823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:06:06.438766 4930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f280bffa41a4b7f8958cf86205b6192" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41895 } }
I20260806 01:06:06.439101 4930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:06.439214 4930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f280bffa41a4b7f8958cf86205b6192, State: Initialized, Role: FOLLOWER
I20260806 01:06:06.439826 4930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f280bffa41a4b7f8958cf86205b6192" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41895 } }
I20260806 01:06:06.440060 4930 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:06.440204 4930 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:06.440376 4930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:06.441421 4934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:06.420423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:51775 (local address 127.0.0.1:41895)
0806 01:06:06.421257 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:06.421270 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:06.421315 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:06:06.422896 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:06.422916 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:06.423278 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:06.423702 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:06.425506 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:06.426776 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:06.428474 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:06.429470 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:06.429618 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:06.434261 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:06.434267 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:06.434279 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:06.434312 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:06.437395 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:06.438108 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:06.438114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:06.438119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:06.438275 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:06.438729 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:06.438733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:06.438735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:06.439286 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:06.439679 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:06.440101 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 01:06:06.440408 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":373,"threads_started":1}
I20260806 01:06:06.442600 4930 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f280bffa41a4b7f8958cf86205b6192" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41895 } }
I20260806 01:06:06.443049 4930 leader_election.cc:302] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f280bffa41a4b7f8958cf86205b6192; no voters:
I20260806 01:06:06.443681 4930 leader_election.cc:288] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:06.443781 4937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:06.444960 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:06.446116 4930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:06.450870 4937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [term 1 LEADER]: Becoming Leader. State: Replica: 4f280bffa41a4b7f8958cf86205b6192, State: Running, Role: LEADER
I20260806 01:06:06.451550 4937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f280bffa41a4b7f8958cf86205b6192" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41895 } }
I20260806 01:06:06.454686 4938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f280bffa41a4b7f8958cf86205b6192" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f280bffa41a4b7f8958cf86205b6192" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41895 } } }
I20260806 01:06:06.455157 4938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:06.467841 4942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f280bffa41a4b7f8958cf86205b6192. Latest consensus state: current_term: 1 leader_uuid: "4f280bffa41a4b7f8958cf86205b6192" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f280bffa41a4b7f8958cf86205b6192" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41895 } } }
I20260806 01:06:06.468246 4942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:06.469051 4957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:06.471637 4934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:06.449251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:47331 (local address 127.0.0.1:41895)
0806 01:06:06.449407 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:06.449413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:06.451944 (+ 2531us) server_negotiation.cc:407] Connection header received
0806 01:06:06.452146 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:06.452157 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:06.452211 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:06.452308 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:06.454023 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:06.455129 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:06.457477 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:06.458350 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:06.458484 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:06.462565 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:06.462575 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:06.462581 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:06.462637 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:06.467069 (+ 4432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:06.468244 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:06.468252 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:06.468258 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:06.468401 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:06.469084 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:06.469092 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:06.469098 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:06.469660 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:06.469965 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:06.470840 (+ 875us) server_negotiation.cc:299] Negotiation successful
0806 01:06:06.471063 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 01:06:06.476292 4957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:06.486874 4957 catalog_manager.cc:1347] Generated new cluster ID: 6aa3b21b08164b72bf61c2f214ebd2b1
I20260806 01:06:06.487005 4957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:06.495326 4957 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:06.496876 4957 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:06.505889 4957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f280bffa41a4b7f8958cf86205b6192: Generated new TSK 0
I20260806 01:06:06.506796 4957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:06.514269 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 4843
I20260806 01:06:06.515412 4923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:06.600167 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 4843
W20260806 01:06:06.623330 4819 connection.cc:535] client connection to 127.0.0.1:41895 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41895): Connection reset by peer (error 104)
2026-08-06T01:06:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2177 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:06 dist-test-slave-x4qh krb5kdc[4984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:06 dist-test-slave-x4qh krb5kdc[4984](info): set up 2 sockets
Aug 06 01:06:06 dist-test-slave-x4qh krb5kdc[4984](info): commencing operation
krb5kdc: starting...
W20260806 01:06:07.892269 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:08 dist-test-slave-x4qh krb5kdc[4984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:06:08.180608 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:08.507573 5003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:08.507810 5003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:08.507962 5003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:08.514624 5003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:08.514756 5003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:08.514853 5003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:08.514953 5003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:08.515015 5003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:08.523942 5003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41297
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:08.525019 5003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:08.526194 5003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:08.537581 5009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:08.537660 5010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:08.537628 5014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:08.538338 5003 server_base.cc:1034] running on GCE node
I20260806 01:06:08.538997 5003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:08.540194 5003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:08.541419 5003 hybrid_clock.cc:648] HybridClock initialized: now 1785978368541387 us; error 38 us; skew 500 ppm
I20260806 01:06:08.541757 5003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:08 dist-test-slave-x4qh krb5kdc[4984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978368, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:08.544696 5003 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:06:08.546162 5003 webserver.cc:460] Webserver started at http://127.0.0.1:45001/ using document root <none> and password file <none>
I20260806 01:06:08.546806 5003 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:08.546969 5003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:08.547262 5003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:08.550125 5003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "4d5773e2b72742de9c7451c04938096e"
format_stamp: "Formatted at 2026-08-06 01:06:08 on dist-test-slave-x4qh"
server_key: "63a5d8f1d485f9c732d6315e7fe3038a"
server_key_iv: "c1627adddeb0a5d0a61427617e1731c1"
server_key_version: "encryptionkey@0"
I20260806 01:06:08.550905 5003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "4d5773e2b72742de9c7451c04938096e"
format_stamp: "Formatted at 2026-08-06 01:06:08 on dist-test-slave-x4qh"
server_key: "63a5d8f1d485f9c732d6315e7fe3038a"
server_key_iv: "c1627adddeb0a5d0a61427617e1731c1"
server_key_version: "encryptionkey@0"
I20260806 01:06:08.555799 5003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:06:08.559041 5023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:08.560222 5003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:06:08.560418 5003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "4d5773e2b72742de9c7451c04938096e"
format_stamp: "Formatted at 2026-08-06 01:06:08 on dist-test-slave-x4qh"
server_key: "63a5d8f1d485f9c732d6315e7fe3038a"
server_key_iv: "c1627adddeb0a5d0a61427617e1731c1"
server_key_version: "encryptionkey@0"
I20260806 01:06:08.560600 5003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:08.577191 5003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:08.580833 5003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:08.581101 5003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:08.598688 5003 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41297
I20260806 01:06:08.598723 5097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41297 every 8 connection(s)
I20260806 01:06:08.600095 5003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:08.603449 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:08.605913 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5003
I20260806 01:06:08.606081 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:08.606647 5554 external_mini_cluster.cc:1428] Setting key 498ff2dbfeafd3ed18fc1b7455c929a0
I20260806 01:06:08.614940 5098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e: Bootstrap starting.
I20260806 01:06:08.618417 5098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:08.619357 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:08.608420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:51941 (local address 127.0.0.1:41297)
0806 01:06:08.609213 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:08.609226 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:08.609264 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:06:08.610830 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:08.610851 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:08.611204 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:08.611569 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:08.612163 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:08.613428 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:08.615152 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:08.615795 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:08.615915 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:08.617239 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:08.617246 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:08.617258 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:08.617759 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:08.617828 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:08.617856 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:08.617968 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:08.618263 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 01:06:08.618459 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":398,"threads_started":1}
I20260806 01:06:08.619542 5098 log.cc:824] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:08.622453 5098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e: No bootstrap required, opened a new log
I20260806 01:06:08.623227 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:08.630183 5098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5773e2b72742de9c7451c04938096e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41297 } }
I20260806 01:06:08.630590 5098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:08.630733 5098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5773e2b72742de9c7451c04938096e, State: Initialized, Role: FOLLOWER
I20260806 01:06:08.631340 5098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5773e2b72742de9c7451c04938096e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41297 } }
I20260806 01:06:08.631660 5098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:08.631834 5098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:08.632047 5098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:08.634203 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:08.627601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:43329 (local address 127.0.0.1:41297)
0806 01:06:08.627792 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:08.627798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:08.628116 (+ 318us) server_negotiation.cc:407] Connection header received
0806 01:06:08.628278 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:08.628281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:08.628338 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:08.628415 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:08.629023 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:08.629745 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:08.631647 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:08.632286 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:08.632397 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:08.633477 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:08.633482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:08.633485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:08.633501 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:08.633533 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:08.633550 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:08.633636 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:08.633827 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 01:06:08.633867 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:06:08.634791 5098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5773e2b72742de9c7451c04938096e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41297 } }
I20260806 01:06:08.635257 5098 leader_election.cc:302] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d5773e2b72742de9c7451c04938096e; no voters:
I20260806 01:06:08.635828 5098 leader_election.cc:288] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:08.635926 5113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:08.636675 5113 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [term 1 LEADER]: Becoming Leader. State: Replica: 4d5773e2b72742de9c7451c04938096e, State: Running, Role: LEADER
I20260806 01:06:08.637312 5113 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5773e2b72742de9c7451c04938096e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41297 } }
I20260806 01:06:08.637959 5098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:08.640053 5114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d5773e2b72742de9c7451c04938096e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5773e2b72742de9c7451c04938096e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41297 } } }
I20260806 01:06:08.640364 5114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:08.642539 5116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d5773e2b72742de9c7451c04938096e. Latest consensus state: current_term: 1 leader_uuid: "4d5773e2b72742de9c7451c04938096e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5773e2b72742de9c7451c04938096e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41297 } } }
I20260806 01:06:08.643061 5116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:08.643251 5120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:08.648396 5120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:08.656961 5120 catalog_manager.cc:1347] Generated new cluster ID: 46545524d4fb48249f9dbc440fe61802
I20260806 01:06:08.657076 5120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:08.687521 5120 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:08.688773 5120 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:08.695077 5120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d5773e2b72742de9c7451c04938096e: Generated new TSK 0
I20260806 01:06:08.695889 5120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:08.715482 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5003
W20260806 01:06:08.716849 5093 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.93:51941
W20260806 01:06:08.717597 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5003: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:08.717747 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5003
W20260806 01:06:08.742698 4982 connection.cc:535] client connection to 127.0.0.1:41297 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41297): Connection reset by peer (error 104)
2026-08-06T01:06:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2119 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:08 dist-test-slave-x4qh krb5kdc[5149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:08 dist-test-slave-x4qh krb5kdc[5149](info): set up 2 sockets
Aug 06 01:06:08 dist-test-slave-x4qh krb5kdc[5149](info): commencing operation
krb5kdc: starting...
W20260806 01:06:10.228881 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.467s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:10 dist-test-slave-x4qh krb5kdc[5149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:06:10.680526 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33953
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:11.014556 5168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:11.014811 5168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:11.014997 5168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:11.021919 5168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:11.022015 5168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:11.022135 5168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:11.022243 5168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:11.022317 5168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:11.031427 5168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33953
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:11.032567 5168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:11.033833 5168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:11.044970 5174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:11.044929 5178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:11.045009 5176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:11.045619 5168 server_base.cc:1034] running on GCE node
I20260806 01:06:11.046223 5168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:11.047407 5168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:11.048641 5168 hybrid_clock.cc:648] HybridClock initialized: now 1785978371048583 us; error 42 us; skew 500 ppm
I20260806 01:06:11.048992 5168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:11 dist-test-slave-x4qh krb5kdc[5149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978371, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:11.056056 5168 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:06:11.057443 5168 webserver.cc:460] Webserver started at http://127.0.0.1:40541/ using document root <none> and password file <none>
I20260806 01:06:11.058068 5168 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:11.058215 5168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:11.058501 5168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:11.061290 5168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "6b0dbcbf5882457ca9d2377a6e308dfc"
format_stamp: "Formatted at 2026-08-06 01:06:11 on dist-test-slave-x4qh"
server_key: "3d8c4449ea0e98464d0080c8e6bc59a6"
server_key_iv: "7d90290e4b3d8db414a273bbea016ecc"
server_key_version: "encryptionkey@0"
I20260806 01:06:11.062041 5168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "6b0dbcbf5882457ca9d2377a6e308dfc"
format_stamp: "Formatted at 2026-08-06 01:06:11 on dist-test-slave-x4qh"
server_key: "3d8c4449ea0e98464d0080c8e6bc59a6"
server_key_iv: "7d90290e4b3d8db414a273bbea016ecc"
server_key_version: "encryptionkey@0"
I20260806 01:06:11.066689 5168 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260806 01:06:11.070956 5187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:11.072219 5168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:06:11.072408 5168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "6b0dbcbf5882457ca9d2377a6e308dfc"
format_stamp: "Formatted at 2026-08-06 01:06:11 on dist-test-slave-x4qh"
server_key: "3d8c4449ea0e98464d0080c8e6bc59a6"
server_key_iv: "7d90290e4b3d8db414a273bbea016ecc"
server_key_version: "encryptionkey@0"
I20260806 01:06:11.072582 5168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:11.077737 5168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:11.081274 5168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:11.081563 5168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:11.101199 5168 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33953
I20260806 01:06:11.101240 5256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33953 every 8 connection(s)
I20260806 01:06:11.102632 5168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:11.106346 5257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:11.106439 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5168
I20260806 01:06:11.106616 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:11.107170 5554 external_mini_cluster.cc:1428] Setting key 17a66e63c024b26c672aaae2cc96738c
I20260806 01:06:11.115298 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:11.108949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:39165 (local address 127.0.0.1:33953)
0806 01:06:11.109829 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:11.109839 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:11.109882 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:06:11.112281 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:11.112302 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:11.112625 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:11.112935 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:11.113885 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:11.113891 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:11.113902 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:11.113916 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:11.113980 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:11.114003 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:11.114177 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:11.114272 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 01:06:11.114453 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":409,"threads_started":1}
I20260806 01:06:11.118397 5257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc: Bootstrap starting.
I20260806 01:06:11.118770 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:11.121940 5257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:11.123304 5257 log.cc:824] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:11.126137 5257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc: No bootstrap required, opened a new log
I20260806 01:06:11.126806 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:11.124053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:58993 (local address 127.0.0.1:33953)
0806 01:06:11.124215 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:11.124222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:11.124807 (+ 585us) server_negotiation.cc:407] Connection header received
0806 01:06:11.125003 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:11.125008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:11.125061 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:11.125170 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:11.126144 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:11.126149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:11.126154 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:11.126173 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:11.126223 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:11.126245 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:11.126356 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:11.126466 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 01:06:11.126532 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 01:06:11.133140 5257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0dbcbf5882457ca9d2377a6e308dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33953 } }
I20260806 01:06:11.133476 5257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:11.133571 5257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbcbf5882457ca9d2377a6e308dfc, State: Initialized, Role: FOLLOWER
I20260806 01:06:11.134054 5257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0dbcbf5882457ca9d2377a6e308dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33953 } }
I20260806 01:06:11.134284 5257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:11.134439 5257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:11.134609 5257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:11.136781 5257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0dbcbf5882457ca9d2377a6e308dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33953 } }
I20260806 01:06:11.137270 5257 leader_election.cc:302] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b0dbcbf5882457ca9d2377a6e308dfc; no voters:
I20260806 01:06:11.137833 5257 leader_election.cc:288] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:11.137969 5272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:11.138669 5272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [term 1 LEADER]: Becoming Leader. State: Replica: 6b0dbcbf5882457ca9d2377a6e308dfc, State: Running, Role: LEADER
I20260806 01:06:11.139204 5272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0dbcbf5882457ca9d2377a6e308dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33953 } }
I20260806 01:06:11.139813 5257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:11.141986 5276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b0dbcbf5882457ca9d2377a6e308dfc. Latest consensus state: current_term: 1 leader_uuid: "6b0dbcbf5882457ca9d2377a6e308dfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0dbcbf5882457ca9d2377a6e308dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33953 } } }
I20260806 01:06:11.142495 5276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:11.143478 5283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:11.143450 5274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b0dbcbf5882457ca9d2377a6e308dfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0dbcbf5882457ca9d2377a6e308dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33953 } } }
I20260806 01:06:11.145068 5274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:11.148819 5283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:11.157223 5283 catalog_manager.cc:1347] Generated new cluster ID: cdef672340e2487e9a685ee81bba9aba
I20260806 01:06:11.157342 5283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:11.166464 5283 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:11.166812 5283 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:11.173133 5283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b0dbcbf5882457ca9d2377a6e308dfc: Generated new TSK 0
I20260806 01:06:11.173985 5283 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:11.187065 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5168
W20260806 01:06:11.188174 5251 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.93:39165
W20260806 01:06:11.188879 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5168: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:11.189034 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5168
2026-08-06T01:06:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:11 dist-test-slave-x4qh krb5kdc[5301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:11 dist-test-slave-x4qh krb5kdc[5301](info): set up 2 sockets
Aug 06 01:06:11 dist-test-slave-x4qh krb5kdc[5301](info): commencing operation
krb5kdc: starting...
W20260806 01:06:12.445797 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:12 dist-test-slave-x4qh krb5kdc[5301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:06:12.720577 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:46017
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:36241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:46017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:13.049942 5319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:13.050184 5319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:13.050269 5319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:13.056932 5319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:13.057044 5319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:13.057089 5319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:13.057166 5319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:13.057235 5319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:13.066215 5319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:36241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:46017
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:46017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:13.067236 5319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:13.068436 5319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:13.080901 5324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:13.080961 5328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:13.080864 5325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:13.081687 5319 server_base.cc:1034] running on GCE node
I20260806 01:06:13.082413 5319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:13.083797 5319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:13.085050 5319 hybrid_clock.cc:648] HybridClock initialized: now 1785978373085003 us; error 33 us; skew 500 ppm
I20260806 01:06:13.085394 5319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:13 dist-test-slave-x4qh krb5kdc[5301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978373, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:13.088676 5319 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:06:13.090054 5319 webserver.cc:460] Webserver started at http://127.5.108.190:45445/ using document root <none> and password file <none>
I20260806 01:06:13.090662 5319 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:13.090799 5319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:13.091104 5319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:13.093955 5319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "7c813dd470924727a74cfea0fe8398b7"
format_stamp: "Formatted at 2026-08-06 01:06:13 on dist-test-slave-x4qh"
server_key: "330c34e016a8ef8c96a33707eb8dd7c3"
server_key_iv: "8eb8d67e97cef89879f4c76ae9851a83"
server_key_version: "encryptionkey@0"
I20260806 01:06:13.094681 5319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "7c813dd470924727a74cfea0fe8398b7"
format_stamp: "Formatted at 2026-08-06 01:06:13 on dist-test-slave-x4qh"
server_key: "330c34e016a8ef8c96a33707eb8dd7c3"
server_key_iv: "8eb8d67e97cef89879f4c76ae9851a83"
server_key_version: "encryptionkey@0"
I20260806 01:06:13.099450 5319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260806 01:06:13.102662 5338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:13.103861 5319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:06:13.104035 5319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "7c813dd470924727a74cfea0fe8398b7"
format_stamp: "Formatted at 2026-08-06 01:06:13 on dist-test-slave-x4qh"
server_key: "330c34e016a8ef8c96a33707eb8dd7c3"
server_key_iv: "8eb8d67e97cef89879f4c76ae9851a83"
server_key_version: "encryptionkey@0"
I20260806 01:06:13.104224 5319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:13.162667 5319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:13.167091 5319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:13.167479 5319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:13.186203 5319 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:46017
I20260806 01:06:13.186226 5413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:46017 every 8 connection(s)
I20260806 01:06:13.187626 5319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:13.189579 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5319
I20260806 01:06:13.189774 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:13.190207 5554 external_mini_cluster.cc:1428] Setting key 19261eca3c82c5a6bc891d2dc1a7fde9
I20260806 01:06:13.191610 5414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:06:13 dist-test-slave-x4qh krb5kdc[5301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:06:13.205241 5414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7: Bootstrap starting.
I20260806 01:06:13.209110 5414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:13.210441 5414 log.cc:824] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:13.213893 5414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7: No bootstrap required, opened a new log
I20260806 01:06:13.215298 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:13.192056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53619 (local address 127.5.108.190:46017)
0806 01:06:13.192852 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:13.192866 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:13.192914 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:06:13.194897 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:13.194923 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:13.195394 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:13.195809 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:13.197195 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:13.199275 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:13.200583 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:13.201639 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:13.201832 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:13.206339 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:13.206349 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:13.206368 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:13.206417 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:13.210502 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:13.211557 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:13.211567 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:13.211572 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:13.211767 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:13.212282 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:13.212286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:13.212289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:13.212949 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:13.213276 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:13.213731 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 01:06:13.214050 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":361,"threads_started":1}
I20260806 01:06:13.219264 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:13.222203 5414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c813dd470924727a74cfea0fe8398b7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46017 } }
I20260806 01:06:13.222558 5414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:13.222676 5414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c813dd470924727a74cfea0fe8398b7, State: Initialized, Role: FOLLOWER
I20260806 01:06:13.223186 5414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c813dd470924727a74cfea0fe8398b7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46017 } }
I20260806 01:06:13.223474 5414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:13.223686 5414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:13.223876 5414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:13.227017 5414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c813dd470924727a74cfea0fe8398b7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46017 } }
I20260806 01:06:13.227533 5414 leader_election.cc:302] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c813dd470924727a74cfea0fe8398b7; no voters:
I20260806 01:06:13.228183 5414 leader_election.cc:288] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:13.228250 5424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:13.229656 5424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [term 1 LEADER]: Becoming Leader. State: Replica: 7c813dd470924727a74cfea0fe8398b7, State: Running, Role: LEADER
I20260806 01:06:13.230126 5414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:13.230263 5424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c813dd470924727a74cfea0fe8398b7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46017 } }
I20260806 01:06:13.233770 5425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c813dd470924727a74cfea0fe8398b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c813dd470924727a74cfea0fe8398b7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46017 } } }
I20260806 01:06:13.234453 5425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:13.237248 5426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c813dd470924727a74cfea0fe8398b7. Latest consensus state: current_term: 1 leader_uuid: "7c813dd470924727a74cfea0fe8398b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c813dd470924727a74cfea0fe8398b7" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 46017 } } }
I20260806 01:06:13.243065 5426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:13.244266 5432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:13.245987 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:13.223771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46085 (local address 127.5.108.190:46017)
0806 01:06:13.223988 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:13.223996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:13.224488 (+ 492us) server_negotiation.cc:407] Connection header received
0806 01:06:13.224666 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:13.224670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:13.224725 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:13.224797 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:13.226591 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:13.227322 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:13.229061 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:13.229769 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:13.229894 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:13.233175 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:13.233182 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:13.233188 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:13.233225 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:13.237450 (+ 4225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:13.238623 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:13.238633 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:13.238639 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:13.243245 (+ 4606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:13.243780 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:13.243785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:13.243787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:13.244138 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:13.244446 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:13.245213 (+ 767us) server_negotiation.cc:299] Negotiation successful
0806 01:06:13.245443 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:06:13.249691 5432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:13.253933 5442 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:06:13.258816 5432 catalog_manager.cc:1347] Generated new cluster ID: 40aea3d9a5c94d159cf8f7ed304b8a64
I20260806 01:06:13.258927 5432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:13.288066 5432 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:13.289201 5432 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:13.301975 5432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c813dd470924727a74cfea0fe8398b7: Generated new TSK 0
I20260806 01:06:13.302752 5432 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:13.310539 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5319
I20260806 01:06:13.311684 5408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:13.399631 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5319
W20260806 01:06:13.426175 5298 connection.cc:535] client connection to 127.5.108.190:46017 recv error: Network error: failed to read from TLS socket (remote: 127.5.108.190:46017): Connection reset by peer (error 104)
2026-08-06T01:06:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:13 dist-test-slave-x4qh krb5kdc[5447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:13 dist-test-slave-x4qh krb5kdc[5447](info): set up 2 sockets
Aug 06 01:06:13 dist-test-slave-x4qh krb5kdc[5447](info): commencing operation
krb5kdc: starting...
W20260806 01:06:14.666587 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.002s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:14 dist-test-slave-x4qh krb5kdc[5447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:06:14.948266 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34855
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:40653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:34855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:15.274446 5479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:15.274703 5479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:15.274824 5479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:15.281810 5479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:15.281917 5479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:15.282027 5479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:15.282131 5479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:15.282212 5479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:15.292088 5479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:40653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:34855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:34855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:15.293370 5479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:15.294708 5479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:15.306564 5487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:15.306633 5489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:15.306602 5493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:15.307322 5479 server_base.cc:1034] running on GCE node
I20260806 01:06:15.308138 5479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:15.309409 5479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:15.310642 5479 hybrid_clock.cc:648] HybridClock initialized: now 1785978375310602 us; error 46 us; skew 500 ppm
I20260806 01:06:15.310986 5479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:15 dist-test-slave-x4qh krb5kdc[5447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978375, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:15.313997 5479 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:06:15.315510 5479 webserver.cc:460] Webserver started at http://127.5.108.190:37251/ using document root <none> and password file <none>
I20260806 01:06:15.316175 5479 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:15.316318 5479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:15.316639 5479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:15.319559 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "9c3396d2ccfa4663b0ebf5b70deaba94"
format_stamp: "Formatted at 2026-08-06 01:06:15 on dist-test-slave-x4qh"
server_key: "73ca16512ba447e56ec325402b4b9275"
server_key_iv: "a8964854365d29ac8c3dfab7c6bc6f26"
server_key_version: "encryptionkey@0"
I20260806 01:06:15.320292 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "9c3396d2ccfa4663b0ebf5b70deaba94"
format_stamp: "Formatted at 2026-08-06 01:06:15 on dist-test-slave-x4qh"
server_key: "73ca16512ba447e56ec325402b4b9275"
server_key_iv: "a8964854365d29ac8c3dfab7c6bc6f26"
server_key_version: "encryptionkey@0"
I20260806 01:06:15.325073 5479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:06:15.328379 5501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:15.329563 5479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:06:15.329766 5479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "9c3396d2ccfa4663b0ebf5b70deaba94"
format_stamp: "Formatted at 2026-08-06 01:06:15 on dist-test-slave-x4qh"
server_key: "73ca16512ba447e56ec325402b4b9275"
server_key_iv: "a8964854365d29ac8c3dfab7c6bc6f26"
server_key_version: "encryptionkey@0"
I20260806 01:06:15.329967 5479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:15.363657 5479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:15.367329 5479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:15.367615 5479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:15.385951 5479 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:34855
I20260806 01:06:15.385978 5576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:34855 every 8 connection(s)
I20260806 01:06:15.387260 5479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:15.390843 5577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:15.393924 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5479
I20260806 01:06:15.394120 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:15.394582 5554 external_mini_cluster.cc:1428] Setting key 59e03c7b018e6dcf44e90f6a0161b85f
I20260806 01:06:15.403808 5577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94: Bootstrap starting.
I20260806 01:06:15.407359 5577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:15.408686 5577 log.cc:824] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:15.409220 5582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:15.396435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33139 (local address 127.5.108.190:34855)
0806 01:06:15.397229 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:15.397244 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:15.397290 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:06:15.399311 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:15.399341 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:15.399876 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:15.400313 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:15.400845 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:15.402882 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:15.404240 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:15.405254 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:15.405468 (+ 214us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:15.406520 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:15.406529 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:15.406546 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:15.407170 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:15.407265 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:15.407296 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:15.407482 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:15.407799 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 01:06:15.407965 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":326,"threads_started":1}
I20260806 01:06:15.411616 5577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94: No bootstrap required, opened a new log
I20260806 01:06:15.412864 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:15.419039 5577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3396d2ccfa4663b0ebf5b70deaba94" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34855 } }
I20260806 01:06:15.419459 5577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:15.419543 5577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3396d2ccfa4663b0ebf5b70deaba94, State: Initialized, Role: FOLLOWER
I20260806 01:06:15.420104 5577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3396d2ccfa4663b0ebf5b70deaba94" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34855 } }
I20260806 01:06:15.420351 5577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:15.420509 5577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:15.420696 5577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:15.422907 5577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3396d2ccfa4663b0ebf5b70deaba94" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34855 } }
I20260806 01:06:15.423223 5582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:15.416916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34161 (local address 127.5.108.190:34855)
0806 01:06:15.417072 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:15.417077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:15.417437 (+ 360us) server_negotiation.cc:407] Connection header received
0806 01:06:15.417559 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:15.417564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:15.417641 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:15.417734 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:15.418147 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:15.419127 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:15.420531 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:15.421059 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:15.421207 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:15.422349 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:15.422355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:15.422361 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:15.422383 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:15.422448 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:15.422463 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:15.422590 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:15.422778 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 01:06:15.422824 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:06:15.423563 5577 leader_election.cc:302] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c3396d2ccfa4663b0ebf5b70deaba94; no voters:
I20260806 01:06:15.424175 5577 leader_election.cc:288] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:15.424309 5590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:15.426184 5577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:15.431486 5590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [term 1 LEADER]: Becoming Leader. State: Replica: 9c3396d2ccfa4663b0ebf5b70deaba94, State: Running, Role: LEADER
I20260806 01:06:15.432147 5590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3396d2ccfa4663b0ebf5b70deaba94" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34855 } }
I20260806 01:06:15.435068 5591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c3396d2ccfa4663b0ebf5b70deaba94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3396d2ccfa4663b0ebf5b70deaba94" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34855 } } }
I20260806 01:06:15.435173 5592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c3396d2ccfa4663b0ebf5b70deaba94. Latest consensus state: current_term: 1 leader_uuid: "9c3396d2ccfa4663b0ebf5b70deaba94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3396d2ccfa4663b0ebf5b70deaba94" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 34855 } } }
I20260806 01:06:15.435489 5591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:15.435590 5592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94 [sys.catalog]: This master's current role is: LEADER
W20260806 01:06:15.440084 5610 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:06:15.440207 5610 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:06:15.440337 5612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:15.442430 5612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:15.450137 5612 catalog_manager.cc:1347] Generated new cluster ID: 19dae7ec377e426b93bbd1cb40aab996
I20260806 01:06:15.450235 5612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:15.466094 5612 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:15.467278 5612 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:15.474470 5612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c3396d2ccfa4663b0ebf5b70deaba94: Generated new TSK 0
I20260806 01:06:15.475111 5612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:15.483131 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5479
W20260806 01:06:15.484683 5572 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33139
W20260806 01:06:15.485360 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5479: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:15.485508 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5479
W20260806 01:06:15.511195 5445 connection.cc:535] client connection to 127.5.108.190:34855 recv error: Network error: failed to read from TLS socket (remote: 127.5.108.190:34855): Connection reset by peer (error 104)
2026-08-06T01:06:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2085 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:15 dist-test-slave-x4qh krb5kdc[5621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:15 dist-test-slave-x4qh krb5kdc[5621](info): set up 2 sockets
Aug 06 01:06:15 dist-test-slave-x4qh krb5kdc[5621](info): commencing operation
krb5kdc: starting...
W20260806 01:06:16.748839 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:16 dist-test-slave-x4qh krb5kdc[5621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:06:17.051837 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36587
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:17.391088 5642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:17.391477 5642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:17.391620 5642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:17.398475 5642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:17.398576 5642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:17.398630 5642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:17.398687 5642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:17.398734 5642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:17.408107 5642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36587
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:17.409276 5642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:17.410512 5642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:17.422602 5650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:17.422538 5651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:17.422664 5655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:17.423712 5642 server_base.cc:1034] running on GCE node
I20260806 01:06:17.424547 5642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:17.425863 5642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:17.427101 5642 hybrid_clock.cc:648] HybridClock initialized: now 1785978377427042 us; error 44 us; skew 500 ppm
I20260806 01:06:17.427521 5642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:17 dist-test-slave-x4qh krb5kdc[5621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978377, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:17.435104 5642 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:06:17.436686 5642 webserver.cc:460] Webserver started at http://127.5.108.190:41377/ using document root <none> and password file <none>
I20260806 01:06:17.437347 5642 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:17.437511 5642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:17.437844 5642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:17.440907 5642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "8840345d59a0473eb1a87c7b447d4cdc"
format_stamp: "Formatted at 2026-08-06 01:06:17 on dist-test-slave-x4qh"
server_key: "6ba9a987b79d7d964d780fa44920f6f9"
server_key_iv: "8ebca840b4acc8efec5f823d45ad264b"
server_key_version: "encryptionkey@0"
I20260806 01:06:17.441774 5642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "8840345d59a0473eb1a87c7b447d4cdc"
format_stamp: "Formatted at 2026-08-06 01:06:17 on dist-test-slave-x4qh"
server_key: "6ba9a987b79d7d964d780fa44920f6f9"
server_key_iv: "8ebca840b4acc8efec5f823d45ad264b"
server_key_version: "encryptionkey@0"
I20260806 01:06:17.446959 5642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260806 01:06:17.450150 5669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:17.451332 5642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:06:17.451607 5642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "8840345d59a0473eb1a87c7b447d4cdc"
format_stamp: "Formatted at 2026-08-06 01:06:17 on dist-test-slave-x4qh"
server_key: "6ba9a987b79d7d964d780fa44920f6f9"
server_key_iv: "8ebca840b4acc8efec5f823d45ad264b"
server_key_version: "encryptionkey@0"
I20260806 01:06:17.451828 5642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:17.457216 5642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:17.461601 5642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:17.461902 5642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:17.481067 5642 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36587
I20260806 01:06:17.481104 5743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36587 every 8 connection(s)
I20260806 01:06:17.482342 5642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:17.485782 5744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:17.491807 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5642
I20260806 01:06:17.492007 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:17.492584 5554 external_mini_cluster.cc:1428] Setting key 418383ad9db757bc6752258e630adcd3
I20260806 01:06:17.495942 5744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc: Bootstrap starting.
I20260806 01:06:17.499410 5744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:17.499971 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:17.494196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50791 (local address 127.5.108.190:36587)
0806 01:06:17.494942 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:17.494952 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:17.494990 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:06:17.496700 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:17.496719 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:17.497011 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:17.497317 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:17.498357 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:17.498363 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:17.498379 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:17.498402 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:17.498477 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:17.498496 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:17.498670 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:17.498794 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 01:06:17.498973 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":321,"threads_started":1}
I20260806 01:06:17.500515 5744 log.cc:824] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:17.503559 5744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc: No bootstrap required, opened a new log
I20260806 01:06:17.503631 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:17.514973 5744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8840345d59a0473eb1a87c7b447d4cdc" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36587 } }
I20260806 01:06:17.515378 5744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:17.515501 5744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8840345d59a0473eb1a87c7b447d4cdc, State: Initialized, Role: FOLLOWER
I20260806 01:06:17.516124 5744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8840345d59a0473eb1a87c7b447d4cdc" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36587 } }
I20260806 01:06:17.516340 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:17.513811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46815 (local address 127.5.108.190:36587)
0806 01:06:17.513966 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:17.513971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:17.513989 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:06:17.514047 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:17.514055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:17.514105 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:17.514196 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:17.515503 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:17.515512 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:17.515518 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:17.515538 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:17.515609 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:17.515635 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:17.515746 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:17.515945 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 01:06:17.516001 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 01:06:17.516527 5744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:17.516676 5744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:17.516865 5744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:17.518823 5744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8840345d59a0473eb1a87c7b447d4cdc" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36587 } }
I20260806 01:06:17.519260 5744 leader_election.cc:302] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8840345d59a0473eb1a87c7b447d4cdc; no voters:
I20260806 01:06:17.519814 5744 leader_election.cc:288] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:17.519894 5755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:17.520560 5755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [term 1 LEADER]: Becoming Leader. State: Replica: 8840345d59a0473eb1a87c7b447d4cdc, State: Running, Role: LEADER
I20260806 01:06:17.521073 5755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8840345d59a0473eb1a87c7b447d4cdc" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36587 } }
I20260806 01:06:17.521675 5744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:17.524096 5760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8840345d59a0473eb1a87c7b447d4cdc. Latest consensus state: current_term: 1 leader_uuid: "8840345d59a0473eb1a87c7b447d4cdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8840345d59a0473eb1a87c7b447d4cdc" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36587 } } }
I20260806 01:06:17.524617 5760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:17.525590 5767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:17.525556 5757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8840345d59a0473eb1a87c7b447d4cdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8840345d59a0473eb1a87c7b447d4cdc" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36587 } } }
I20260806 01:06:17.525898 5757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:17.530521 5767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:17.539610 5767 catalog_manager.cc:1347] Generated new cluster ID: f804007927f7401594e1368bec591dc6
I20260806 01:06:17.539706 5767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:17.565302 5767 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:17.565711 5767 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:17.576385 5767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8840345d59a0473eb1a87c7b447d4cdc: Generated new TSK 0
I20260806 01:06:17.577064 5767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:17.582324 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5642
W20260806 01:06:17.583523 5739 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50791
W20260806 01:06:17.584251 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5642: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:17.584396 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5642
2026-08-06T01:06:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2100 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:17 dist-test-slave-x4qh krb5kdc[5786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:17 dist-test-slave-x4qh krb5kdc[5786](info): set up 2 sockets
Aug 06 01:06:17 dist-test-slave-x4qh krb5kdc[5786](info): commencing operation
krb5kdc: starting...
W20260806 01:06:18.850553 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:19 dist-test-slave-x4qh krb5kdc[5786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:06:19.148490 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36609
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:36609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:19.478698 5805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:19.478977 5805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:19.479154 5805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:19.486132 5805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:19.486229 5805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:19.486330 5805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:19.486423 5805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:19.486481 5805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:19.495726 5805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:36609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:36609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:19.496891 5805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:19.498160 5805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:19.510635 5811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:19.510605 5812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:19.511547 5818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:19.512943 5805 server_base.cc:1034] running on GCE node
I20260806 01:06:19.513896 5805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:19.515297 5805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:19.516552 5805 hybrid_clock.cc:648] HybridClock initialized: now 1785978379516516 us; error 28 us; skew 500 ppm
I20260806 01:06:19.516893 5805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:19 dist-test-slave-x4qh krb5kdc[5786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:19.519887 5805 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:06:19.521204 5805 webserver.cc:460] Webserver started at http://127.5.108.190:36879/ using document root <none> and password file <none>
I20260806 01:06:19.521878 5805 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:19.522040 5805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:19.522349 5805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:19.525413 5805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "52d1e80047b34dafbd5c298922b2d92c"
format_stamp: "Formatted at 2026-08-06 01:06:19 on dist-test-slave-x4qh"
server_key: "eec5cd41edab1514b0a3ede728628f7e"
server_key_iv: "55b1b2a18cb9b0aa777f688ec8cac4de"
server_key_version: "encryptionkey@0"
I20260806 01:06:19.526232 5805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "52d1e80047b34dafbd5c298922b2d92c"
format_stamp: "Formatted at 2026-08-06 01:06:19 on dist-test-slave-x4qh"
server_key: "eec5cd41edab1514b0a3ede728628f7e"
server_key_iv: "55b1b2a18cb9b0aa777f688ec8cac4de"
server_key_version: "encryptionkey@0"
I20260806 01:06:19.531919 5805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260806 01:06:19.535051 5833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:19.536262 5805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 01:06:19.536448 5805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "52d1e80047b34dafbd5c298922b2d92c"
format_stamp: "Formatted at 2026-08-06 01:06:19 on dist-test-slave-x4qh"
server_key: "eec5cd41edab1514b0a3ede728628f7e"
server_key_iv: "55b1b2a18cb9b0aa777f688ec8cac4de"
server_key_version: "encryptionkey@0"
I20260806 01:06:19.536623 5805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:19.562570 5805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:19.566274 5805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:19.566565 5805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:19.585007 5805 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:36609
I20260806 01:06:19.585045 5903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:36609 every 8 connection(s)
I20260806 01:06:19.586313 5805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:19.589875 5904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:19.589991 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5805
I20260806 01:06:19.590184 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:19.590731 5554 external_mini_cluster.cc:1428] Setting key c4efe76bc7813f3e9a89c7cd0248a554
I20260806 01:06:19.604169 5904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c: Bootstrap starting.
Aug 06 01:06:19 dist-test-slave-x4qh krb5kdc[5786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:06:19.608052 5904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:19.609124 5904 log.cc:824] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:19.611881 5904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c: No bootstrap required, opened a new log
I20260806 01:06:19.613341 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:19.592491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45371 (local address 127.5.108.190:36609)
0806 01:06:19.593315 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:19.593330 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:19.593382 (+ 52us) server_negotiation.cc:407] Connection header received
0806 01:06:19.594970 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:19.594994 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:19.595481 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:19.595874 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:19.597265 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:19.599245 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:19.600542 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:19.601591 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:19.601809 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:19.606226 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:19.606233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:19.606245 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:19.606276 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:19.609488 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:19.610165 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:19.610172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:19.610177 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:19.610334 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:19.610734 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:19.610740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:19.610743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:19.611233 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:19.611575 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:19.612030 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 01:06:19.612428 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":384,"threads_started":1}
I20260806 01:06:19.616894 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:19.619846 5904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d1e80047b34dafbd5c298922b2d92c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36609 } }
I20260806 01:06:19.620200 5904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:19.620318 5904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d1e80047b34dafbd5c298922b2d92c, State: Initialized, Role: FOLLOWER
I20260806 01:06:19.620939 5904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d1e80047b34dafbd5c298922b2d92c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36609 } }
I20260806 01:06:19.621244 5904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:19.621512 5904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:19.621703 5904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:19.623924 5904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d1e80047b34dafbd5c298922b2d92c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36609 } }
I20260806 01:06:19.624409 5904 leader_election.cc:302] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52d1e80047b34dafbd5c298922b2d92c; no voters:
I20260806 01:06:19.625056 5904 leader_election.cc:288] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:19.625160 5915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:19.627447 5904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:19.631655 5915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [term 1 LEADER]: Becoming Leader. State: Replica: 52d1e80047b34dafbd5c298922b2d92c, State: Running, Role: LEADER
I20260806 01:06:19.632438 5915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d1e80047b34dafbd5c298922b2d92c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36609 } }
I20260806 01:06:19.634800 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:19.621289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43109 (local address 127.5.108.190:36609)
0806 01:06:19.621488 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:19.621501 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:19.621912 (+ 411us) server_negotiation.cc:407] Connection header received
0806 01:06:19.622101 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:19.622106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:19.622174 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:19.622292 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:19.623744 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:19.624478 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:19.625968 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:19.626479 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:19.626549 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:19.628999 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:19.629004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:19.629007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:19.629034 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:19.632125 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:19.632739 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:19.632745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:19.632748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:19.632878 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:19.633271 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:19.633276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:19.633279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:19.633662 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:19.633915 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:19.634242 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 01:06:19.634427 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:06:19.637604 5916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52d1e80047b34dafbd5c298922b2d92c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d1e80047b34dafbd5c298922b2d92c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36609 } } }
I20260806 01:06:19.637781 5918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52d1e80047b34dafbd5c298922b2d92c. Latest consensus state: current_term: 1 leader_uuid: "52d1e80047b34dafbd5c298922b2d92c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d1e80047b34dafbd5c298922b2d92c" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 36609 } } }
I20260806 01:06:19.638087 5916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:19.638129 5918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:19.640543 5927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:19.645205 5927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:19.653040 5927 catalog_manager.cc:1347] Generated new cluster ID: a687af007cba401bbc6016b304c6072e
I20260806 01:06:19.653162 5927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:19.660037 5927 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:19.661180 5927 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:19.670838 5927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52d1e80047b34dafbd5c298922b2d92c: Generated new TSK 0
I20260806 01:06:19.671710 5927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:19.679199 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5805
I20260806 01:06:19.680228 5899 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:19.765847 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5805
2026-08-06T01:06:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2179 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:19 dist-test-slave-x4qh krb5kdc[5951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:19 dist-test-slave-x4qh krb5kdc[5951](info): set up 2 sockets
Aug 06 01:06:19 dist-test-slave-x4qh krb5kdc[5951](info): commencing operation
krb5kdc: starting...
W20260806 01:06:21.029112 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:21 dist-test-slave-x4qh krb5kdc[5951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:06:21.310380 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:41359
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:41359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:21.647231 5970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:21.647589 5970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:21.647781 5970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:21.654803 5970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:21.654892 5970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:21.654934 5970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:21.655010 5970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:21.655087 5970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:21.664144 5970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:41359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:41359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:21.665285 5970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:21.666476 5970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:21.677932 5980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:21.677994 5979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:21.677971 5984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:21.678984 5970 server_base.cc:1034] running on GCE node
I20260806 01:06:21.679764 5970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:21.680929 5970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:21.682135 5970 hybrid_clock.cc:648] HybridClock initialized: now 1785978381682080 us; error 46 us; skew 500 ppm
I20260806 01:06:21.682497 5970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:21 dist-test-slave-x4qh krb5kdc[5951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978381, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:21.685593 5970 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:06:21.687008 5970 webserver.cc:460] Webserver started at http://127.5.108.190:38419/ using document root <none> and password file <none>
I20260806 01:06:21.687686 5970 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:21.687812 5970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:21.688097 5970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:21.690922 5970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "f4c89905137c4c7a9638d7197f90a7f8"
format_stamp: "Formatted at 2026-08-06 01:06:21 on dist-test-slave-x4qh"
server_key: "a4fec15261c58885fc63d3718be4bfb9"
server_key_iv: "26d1dc6ba83d55bb9d4ba9a47ec0e919"
server_key_version: "encryptionkey@0"
I20260806 01:06:21.691720 5970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "f4c89905137c4c7a9638d7197f90a7f8"
format_stamp: "Formatted at 2026-08-06 01:06:21 on dist-test-slave-x4qh"
server_key: "a4fec15261c58885fc63d3718be4bfb9"
server_key_iv: "26d1dc6ba83d55bb9d4ba9a47ec0e919"
server_key_version: "encryptionkey@0"
I20260806 01:06:21.696401 5970 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:06:21.699596 5993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:21.700783 5970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:06:21.700953 5970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "f4c89905137c4c7a9638d7197f90a7f8"
format_stamp: "Formatted at 2026-08-06 01:06:21 on dist-test-slave-x4qh"
server_key: "a4fec15261c58885fc63d3718be4bfb9"
server_key_iv: "26d1dc6ba83d55bb9d4ba9a47ec0e919"
server_key_version: "encryptionkey@0"
I20260806 01:06:21.701143 5970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:21.733142 5970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:21.736723 5970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:21.736991 5970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:21.756038 5970 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:41359
I20260806 01:06:21.756068 6069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:41359 every 8 connection(s)
I20260806 01:06:21.757323 5970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:21.760041 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 5970
I20260806 01:06:21.760241 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:21.760759 5554 external_mini_cluster.cc:1428] Setting key 8ed4eb784befa2afd649f95ba1ce9593
I20260806 01:06:21.761343 6070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:21.773092 6070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8: Bootstrap starting.
I20260806 01:06:21.773808 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:21.762591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37065 (local address 127.5.108.190:41359)
0806 01:06:21.763365 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:21.763420 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:21.763468 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:06:21.765811 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:21.765830 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:21.766166 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:21.766526 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:21.767059 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:21.768309 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:21.769504 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:21.770144 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:21.770349 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:21.771614 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:21.771621 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:21.771633 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:21.772163 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:21.772238 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:21.772263 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:21.772391 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:21.772616 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 01:06:21.772874 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":343,"threads_started":1}
I20260806 01:06:21.776053 6070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:21.777163 6070 log.cc:824] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:21.777302 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:21.779707 6070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8: No bootstrap required, opened a new log
I20260806 01:06:21.786862 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:21.781231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47803 (local address 127.5.108.190:41359)
0806 01:06:21.781397 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:21.781404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:21.781764 (+ 360us) server_negotiation.cc:407] Connection header received
0806 01:06:21.781988 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:21.781992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:21.782053 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:21.782176 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:21.782641 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:21.783339 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:21.784591 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:21.785024 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:21.785198 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:21.786177 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:21.786181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:21.786184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:21.786197 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:21.786232 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:21.786245 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:21.786345 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:21.786472 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 01:06:21.786567 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 01:06:21.787408 6070 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c89905137c4c7a9638d7197f90a7f8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 41359 } }
I20260806 01:06:21.787784 6070 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:21.787894 6070 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c89905137c4c7a9638d7197f90a7f8, State: Initialized, Role: FOLLOWER
I20260806 01:06:21.788403 6070 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c89905137c4c7a9638d7197f90a7f8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 41359 } }
I20260806 01:06:21.788655 6070 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:21.788790 6070 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:21.788957 6070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:21.790784 6070 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c89905137c4c7a9638d7197f90a7f8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 41359 } }
I20260806 01:06:21.791224 6070 leader_election.cc:302] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4c89905137c4c7a9638d7197f90a7f8; no voters:
I20260806 01:06:21.791740 6070 leader_election.cc:288] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:21.791909 6086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:21.792596 6086 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [term 1 LEADER]: Becoming Leader. State: Replica: f4c89905137c4c7a9638d7197f90a7f8, State: Running, Role: LEADER
I20260806 01:06:21.793159 6086 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c89905137c4c7a9638d7197f90a7f8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 41359 } }
I20260806 01:06:21.793778 6070 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:21.796075 6088 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4c89905137c4c7a9638d7197f90a7f8. Latest consensus state: current_term: 1 leader_uuid: "f4c89905137c4c7a9638d7197f90a7f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c89905137c4c7a9638d7197f90a7f8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 41359 } } }
I20260806 01:06:21.796506 6088 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:21.797070 6087 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4c89905137c4c7a9638d7197f90a7f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c89905137c4c7a9638d7197f90a7f8" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 41359 } } }
I20260806 01:06:21.797482 6087 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:21.800716 6095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:21.806687 6095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:21.816241 6095 catalog_manager.cc:1347] Generated new cluster ID: 11dedea993e74d77b17f0ec41b942f78
I20260806 01:06:21.816341 6095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:21.835103 6095 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:21.836789 6095 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:21.846643 6095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4c89905137c4c7a9638d7197f90a7f8: Generated new TSK 0
I20260806 01:06:21.847482 6095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:21.863468 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5970
W20260806 01:06:21.864459 6064 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37065
W20260806 01:06:21.865134 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 5970: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:21.865281 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 5970
2026-08-06T01:06:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2099 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:21 dist-test-slave-x4qh krb5kdc[6111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:21 dist-test-slave-x4qh krb5kdc[6111](info): set up 2 sockets
Aug 06 01:06:21 dist-test-slave-x4qh krb5kdc[6111](info): commencing operation
krb5kdc: starting...
W20260806 01:06:23.114334 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:23 dist-test-slave-x4qh krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:06:23.410187 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37227
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:37227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:23.731967 6127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:23.732232 6127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:23.732426 6127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:23.738871 6127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:23.738958 6127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:23.739001 6127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:23.739090 6127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:23.739156 6127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:23.748510 6127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:37227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:37227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:23.749657 6127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:23.750892 6127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:23.762244 6140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:23.762466 6137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:23.762430 6136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:23.763510 6127 server_base.cc:1034] running on GCE node
I20260806 01:06:23.764205 6127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:23.765424 6127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:23.766651 6127 hybrid_clock.cc:648] HybridClock initialized: now 1785978383766623 us; error 42 us; skew 500 ppm
I20260806 01:06:23.766978 6127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:23 dist-test-slave-x4qh krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978383, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:23.775566 6127 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:06:23.776978 6127 webserver.cc:460] Webserver started at http://127.5.108.190:41399/ using document root <none> and password file <none>
I20260806 01:06:23.777658 6127 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:23.777812 6127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:23.778115 6127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:23.780974 6127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "e3261914611247cd86ffb1b95d043275"
format_stamp: "Formatted at 2026-08-06 01:06:23 on dist-test-slave-x4qh"
server_key: "f4fa843a0fa10cf7a86ae2e6b085da9d"
server_key_iv: "b653c9c3b07325e0968a8f87c2f68e50"
server_key_version: "encryptionkey@0"
I20260806 01:06:23.781738 6127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "e3261914611247cd86ffb1b95d043275"
format_stamp: "Formatted at 2026-08-06 01:06:23 on dist-test-slave-x4qh"
server_key: "f4fa843a0fa10cf7a86ae2e6b085da9d"
server_key_iv: "b653c9c3b07325e0968a8f87c2f68e50"
server_key_version: "encryptionkey@0"
I20260806 01:06:23.786659 6127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:06:23.789973 6147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:23.791044 6127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:06:23.791214 6127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "e3261914611247cd86ffb1b95d043275"
format_stamp: "Formatted at 2026-08-06 01:06:23 on dist-test-slave-x4qh"
server_key: "f4fa843a0fa10cf7a86ae2e6b085da9d"
server_key_iv: "b653c9c3b07325e0968a8f87c2f68e50"
server_key_version: "encryptionkey@0"
I20260806 01:06:23.791471 6127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:23.797062 6127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:23.800633 6127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:23.800935 6127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:23.819288 6127 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:37227
I20260806 01:06:23.819330 6220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:37227 every 8 connection(s)
I20260806 01:06:23.820598 6127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:23.824261 6221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:23.824935 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6127
I20260806 01:06:23.825129 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:23.825569 5554 external_mini_cluster.cc:1428] Setting key ded0ae10258b26dd8240c8cc9aaff0b7
I20260806 01:06:23.835151 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:23.827447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45761 (local address 127.5.108.190:37227)
0806 01:06:23.828311 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:23.828327 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:23.828397 (+ 70us) server_negotiation.cc:407] Connection header received
0806 01:06:23.831288 (+ 2891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:23.831314 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:23.831776 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:23.832253 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:23.833254 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:23.833263 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:23.833281 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:23.833298 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:23.833363 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:23.833389 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:23.833610 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:23.833748 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 01:06:23.833923 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":336,"threads_started":1}
I20260806 01:06:23.838826 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:23.839650 6221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275: Bootstrap starting.
I20260806 01:06:23.842888 6221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:23.844309 6221 log.cc:824] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:23.846112 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:23.843265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34451 (local address 127.5.108.190:37227)
0806 01:06:23.843453 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:23.843456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:23.843924 (+ 468us) server_negotiation.cc:407] Connection header received
0806 01:06:23.844092 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:23.844097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:23.844149 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:23.844243 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:23.845352 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:23.845356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:23.845359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:06:23.845379 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:23.845424 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:06:23.845444 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:23.845549 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:23.845748 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 01:06:23.845809 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 01:06:23.847767 6221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275: No bootstrap required, opened a new log
I20260806 01:06:23.855505 6221 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3261914611247cd86ffb1b95d043275" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37227 } }
I20260806 01:06:23.855887 6221 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:23.856014 6221 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3261914611247cd86ffb1b95d043275, State: Initialized, Role: FOLLOWER
I20260806 01:06:23.856593 6221 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3261914611247cd86ffb1b95d043275" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37227 } }
I20260806 01:06:23.856843 6221 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:23.857003 6221 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:23.857191 6221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:23.859652 6221 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3261914611247cd86ffb1b95d043275" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37227 } }
I20260806 01:06:23.860193 6221 leader_election.cc:302] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3261914611247cd86ffb1b95d043275; no voters:
I20260806 01:06:23.860786 6221 leader_election.cc:288] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:23.860936 6234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:23.862157 6234 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [term 1 LEADER]: Becoming Leader. State: Replica: e3261914611247cd86ffb1b95d043275, State: Running, Role: LEADER
I20260806 01:06:23.862663 6234 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3261914611247cd86ffb1b95d043275" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37227 } }
I20260806 01:06:23.863018 6221 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:23.865975 6235 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3261914611247cd86ffb1b95d043275" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3261914611247cd86ffb1b95d043275" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37227 } } }
I20260806 01:06:23.871896 6235 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:23.865825 6237 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3261914611247cd86ffb1b95d043275. Latest consensus state: current_term: 1 leader_uuid: "e3261914611247cd86ffb1b95d043275" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3261914611247cd86ffb1b95d043275" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 37227 } } }
I20260806 01:06:23.872547 6237 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:23.872884 6245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:23.878209 6245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:23.888317 6245 catalog_manager.cc:1347] Generated new cluster ID: 3a103205d0654d1ba4f6d4a584e6d984
I20260806 01:06:23.888451 6245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:23.898658 6245 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:23.899024 6245 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:23.910322 6245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3261914611247cd86ffb1b95d043275: Generated new TSK 0
I20260806 01:06:23.911129 6245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:23.926079 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6127
W20260806 01:06:23.927699 6215 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45761
W20260806 01:06:23.928365 5554 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6127: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:23.928498 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6127
2026-08-06T01:06:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2058 ms)
[----------] 18 tests from AuthTokenIssuingTest (38527 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:24 dist-test-slave-x4qh krb5kdc[6263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:24 dist-test-slave-x4qh krb5kdc[6263](info): set up 2 sockets
Aug 06 01:06:24 dist-test-slave-x4qh krb5kdc[6263](info): commencing operation
krb5kdc: starting...
W20260806 01:06:25.177371 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:25 dist-test-slave-x4qh krb5kdc[6263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:06:25.466430 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:39399
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:39399,127.5.108.189:34693,127.5.108.188:40753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:25.797089 6283 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:06:25.797444 6283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:25.797554 6283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:25.797689 6283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:25.804738 6283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:25.804821 6283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:25.804919 6283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:25.804999 6283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:25.805114 6283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:25.814440 6283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:39399,127.5.108.189:34693,127.5.108.188:40753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:39399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:25.815618 6283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:25.816922 6283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:25.828939 6290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:25.829025 6291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:25.828881 6295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:25.829599 6283 server_base.cc:1034] running on GCE node
I20260806 01:06:25.830333 6283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:25.831588 6283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:25.832826 6283 hybrid_clock.cc:648] HybridClock initialized: now 1785978385832771 us; error 32 us; skew 500 ppm
I20260806 01:06:25.833153 6283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:25 dist-test-slave-x4qh krb5kdc[6263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978385, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:25.836190 6283 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:06:25.837592 6283 webserver.cc:460] Webserver started at http://127.5.108.190:39157/ using document root <none> and password file <none>
I20260806 01:06:25.838223 6283 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:25.838366 6283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:25.838727 6283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:25.841727 6283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "7ca8328343ae4b21a126cef2e3c98b33"
format_stamp: "Formatted at 2026-08-06 01:06:25 on dist-test-slave-x4qh"
server_key: "ec935cb128711627e186ae5ce848be4e"
server_key_iv: "4db55de8c7a588c92e234baf0ce8602a"
server_key_version: "encryptionkey@0"
I20260806 01:06:25.842504 6283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "7ca8328343ae4b21a126cef2e3c98b33"
format_stamp: "Formatted at 2026-08-06 01:06:25 on dist-test-slave-x4qh"
server_key: "ec935cb128711627e186ae5ce848be4e"
server_key_iv: "4db55de8c7a588c92e234baf0ce8602a"
server_key_version: "encryptionkey@0"
I20260806 01:06:25.847522 6283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 01:06:25.850754 6306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:25.851945 6283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 01:06:25.852103 6283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "7ca8328343ae4b21a126cef2e3c98b33"
format_stamp: "Formatted at 2026-08-06 01:06:25 on dist-test-slave-x4qh"
server_key: "ec935cb128711627e186ae5ce848be4e"
server_key_iv: "4db55de8c7a588c92e234baf0ce8602a"
server_key_version: "encryptionkey@0"
I20260806 01:06:25.852327 6283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:25.873335 6283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:25.877002 6283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:25.877269 6283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:25.895044 6283 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:39399
I20260806 01:06:25.895082 6386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:39399 every 8 connection(s)
I20260806 01:06:25.896294 6283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:25.899693 6387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:25.901640 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6283
I20260806 01:06:25.901839 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:25.902493 5554 external_mini_cluster.cc:1428] Setting key c6b9769b025b3c0dcbac8476c2629464
I20260806 01:06:25.906055 6387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:06:25 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978385, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:06:25.927757 6392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:25.909023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.5.108.190:39399)
0806 01:06:25.910526 (+ 1503us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:25.910535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:25.910585 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:06:25.912012 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:25.912021 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:25.912288 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:25.912506 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:25.913639 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:25.915358 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:25.916625 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:25.917464 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:25.917595 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:25.921260 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:25.921267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:25.921277 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:25.921305 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:25.923954 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:25.924624 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:25.924637 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:25.924640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:25.924773 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:25.925139 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:25.925145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:25.925148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:25.925552 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:25.925748 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:25.926190 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 01:06:25.926467 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1295,"spinlock_wait_cycles":75136,"thread_start_us":1104,"threads_started":1}
I20260806 01:06:25.927810 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:25.909037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:39399 (local address 127.0.0.1:42580)
0806 01:06:25.909890 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:25.909970 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:25.911688 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:25.912543 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:25.912548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:25.913096 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:25.913425 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:25.913453 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:25.915515 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:25.915519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:25.916485 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:25.916496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:25.917736 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:25.917745 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:25.917952 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:25.918586 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:25.918621 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:25.921056 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:25.924080 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:25.924092 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:25.924102 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:25.924497 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:25.924883 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:25.924888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:25.924892 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:25.925039 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:25.925686 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:25.925697 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:25.925881 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 01:06:25.926084 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 01:06:25.926596 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":123,"thread_start_us":342,"threads_started":1}
I20260806 01:06:25.930328 6387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:06:25.931318 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:25.930985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:40950)
0806 01:06:25.931128 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:25.931224 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:06:25.931576 6309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
I20260806 01:06:25.933032 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:25.932699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:40966)
0806 01:06:25.932849 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:25.932916 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:06:25.933249 6387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:34693: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.108.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab.
Entry for principal kudu/127.5.108.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab.
I20260806 01:06:25.970144 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.189:34693
--webserver_interface=127.5.108.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab
--principal=kudu/127.5.108.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:39399,127.5.108.189:34693,127.5.108.188:40753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:06:25.982535 6387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } attempt: 1
I20260806 01:06:25.983870 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:25.983174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:40974)
0806 01:06:25.983446 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:25.983657 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 01:06:25.985100 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:25.984573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:40986)
0806 01:06:25.984787 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:25.984921 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 01:06:25.985358 6387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:34693: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
I20260806 01:06:26.053622 6387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } attempt: 2
I20260806 01:06:26.054888 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.054311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:41002)
0806 01:06:26.054580 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.054703 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 01:06:26.056000 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.055590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:41016)
0806 01:06:26.055755 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.055838 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:06:26.056205 6387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:34693: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
I20260806 01:06:26.147504 6387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } attempt: 3
I20260806 01:06:26.148801 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.148251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:41022)
0806 01:06:26.148493 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.148617 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 01:06:26.149894 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.149460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:41026)
0806 01:06:26.149625 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.149743 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:06:26.150115 6387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:34693: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
I20260806 01:06:26.293402 6387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } attempt: 4
I20260806 01:06:26.294699 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.294087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:41034)
0806 01:06:26.294423 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.294528 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260806 01:06:26.295770 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.295327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:41042)
0806 01:06:26.295512 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.295610 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 01:06:26.295970 6387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:34693: Network error: Client connection negotiation failed: client connection to 127.5.108.189:34693: connect: Connection refused (error 111)
W20260806 01:06:26.300629 6396 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:06:26.300908 6396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:26.300978 6396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:26.301056 6396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:26.308048 6396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:26.308120 6396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:26.308174 6396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:26.308221 6396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:26.308271 6396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:26.317104 6396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:39399,127.5.108.189:34693,127.5.108.188:40753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab
--principal=kudu/127.5.108.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.189:34693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.108.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:26.318151 6396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:26.319285 6396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:26.330603 6402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:26.330531 6405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:26.330539 6403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:26.331351 6396 server_base.cc:1034] running on GCE node
I20260806 01:06:26.332118 6396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:26.333455 6396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:26.334662 6396 hybrid_clock.cc:648] HybridClock initialized: now 1785978386334593 us; error 49 us; skew 500 ppm
I20260806 01:06:26.334959 6396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:26.337877 6396 init.cc:377] Logged in from keytab as kudu/127.5.108.189@KRBTEST.COM (short username kudu)
I20260806 01:06:26.339160 6396 webserver.cc:460] Webserver started at http://127.5.108.189:45475/ using document root <none> and password file <none>
I20260806 01:06:26.339844 6396 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:26.339967 6396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:26.340271 6396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:26.342906 6396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/data/instance:
uuid: "4765f5e824cc4c3295a732716e85af1d"
format_stamp: "Formatted at 2026-08-06 01:06:26 on dist-test-slave-x4qh"
server_key: "5d7ee26a935fa5d6d93665bc07e3aec4"
server_key_iv: "bfd3e375370fb2a4dbb98987670390b1"
server_key_version: "encryptionkey@0"
I20260806 01:06:26.343811 6396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/wal/instance:
uuid: "4765f5e824cc4c3295a732716e85af1d"
format_stamp: "Formatted at 2026-08-06 01:06:26 on dist-test-slave-x4qh"
server_key: "5d7ee26a935fa5d6d93665bc07e3aec4"
server_key_iv: "bfd3e375370fb2a4dbb98987670390b1"
server_key_version: "encryptionkey@0"
I20260806 01:06:26.348487 6396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:06:26.352313 6419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:26.353574 6396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:06:26.353765 6396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/wal
uuid: "4765f5e824cc4c3295a732716e85af1d"
format_stamp: "Formatted at 2026-08-06 01:06:26 on dist-test-slave-x4qh"
server_key: "5d7ee26a935fa5d6d93665bc07e3aec4"
server_key_iv: "bfd3e375370fb2a4dbb98987670390b1"
server_key_version: "encryptionkey@0"
I20260806 01:06:26.353987 6396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:26.365573 6396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:26.369163 6396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:26.369359 6396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:26.387145 6396 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.189:34693
I20260806 01:06:26.387166 6491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.189:34693 every 8 connection(s)
I20260806 01:06:26.388430 6396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
I20260806 01:06:26.391939 6492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:26.398154 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6396
I20260806 01:06:26.398366 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-1/wal/instance
I20260806 01:06:26.397831 6492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:06:26.398867 5554 external_mini_cluster.cc:1428] Setting key 7754c840b9758ffcf31c4f962dc984ee
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:06:26.421272 6392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.400997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.5.108.190:39399)
0806 01:06:26.401233 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:26.401239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:26.402363 (+ 1124us) server_negotiation.cc:407] Connection header received
0806 01:06:26.405176 (+ 2813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:26.405180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:26.405249 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:26.405339 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:26.407284 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.407960 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.410213 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.410600 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.410694 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.414623 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:26.414627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:26.414630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:26.414669 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:26.418031 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.418690 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.418698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.418703 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.418846 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.419222 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.419230 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.419235 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.419745 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:26.419943 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:26.420554 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 01:06:26.420820 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:06:26.421764 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.401368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:39399 (local address 127.0.0.1:42582)
0806 01:06:26.402142 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.402201 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:26.404858 (+ 2657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:26.405589 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:26.405601 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:26.406358 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:26.407118 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.407142 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.408110 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.408117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.410048 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.410065 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.410819 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.410828 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.411122 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.411808 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:26.411840 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:26.414401 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:26.418163 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.418169 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.418179 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.418558 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.418948 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.418951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.418953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.419112 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.419879 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:26.419884 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:26.420179 (+ 295us) client_negotiation.cc:769] Sending connection context
0806 01:06:26.420403 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 01:06:26.420723 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":305,"threads_started":1}
I20260806 01:06:26.423645 6492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.108.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.188@KRBTEST.COM" created.
I20260806 01:06:26.437700 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.424467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:41046)
0806 01:06:26.424612 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.424633 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:26.424730 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:26.425522 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:26.425526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:26.425943 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:26.426048 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.426055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.427302 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.427307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.428147 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.428158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.428949 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.428954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.429182 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.429703 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:26.429718 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:26.432262 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:26.435360 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.435364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.435401 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.435768 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.436093 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.436096 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.436098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.436236 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.436862 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:26.436866 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:26.437025 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 01:06:26.437194 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 01:06:26.437446 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":6}
I20260806 01:06:26.438297 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.424596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41046 (local address 127.5.108.189:34693)
0806 01:06:26.425170 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:26.425175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:26.425197 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:06:26.425234 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:26.425241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:26.425386 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:26.425527 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:26.426208 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.427161 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.428329 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.428814 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.428943 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.432486 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:26.432497 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:26.432508 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:26.432535 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:26.435150 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.435880 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.435885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.435887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.435992 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.436359 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.436366 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.436372 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.436711 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:26.436925 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:26.437368 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 01:06:26.437618 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":440,"thread_start_us":279,"threads_started":1}
I20260806 01:06:26.440424 6492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:06:26.441411 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.441017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36378)
0806 01:06:26.441182 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.441275 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:06:26.441706 6421 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
I20260806 01:06:26.443151 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.442863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36382)
0806 01:06:26.442988 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.443060 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:06:26.443485 6492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:40753: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab.
Entry for principal kudu/127.5.108.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab.
I20260806 01:06:26.470270 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.188:40753
--webserver_interface=127.5.108.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab
--principal=kudu/127.5.108.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:39399,127.5.108.189:34693,127.5.108.188:40753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:06:26.492802 6492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } attempt: 1
I20260806 01:06:26.493966 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.493551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36390)
0806 01:06:26.493737 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.493829 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 01:06:26.494923 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.494631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36398)
0806 01:06:26.494770 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.494834 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:06:26.495146 6492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:40753: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
I20260806 01:06:26.563424 6492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } attempt: 2
I20260806 01:06:26.564694 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.564179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36402)
0806 01:06:26.564397 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.564489 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 01:06:26.565706 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.565385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36406)
0806 01:06:26.565518 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.565585 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 01:06:26.565912 6492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:40753: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
I20260806 01:06:26.595209 6387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } attempt: 5
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978385, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
I20260806 01:06:26.609321 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.595973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:41048)
0806 01:06:26.596198 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.596215 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:26.596332 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:26.596673 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:26.596677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:26.597024 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:26.597233 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.597242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.598617 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.598620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.599508 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.599521 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.600576 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.600584 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.600804 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.601434 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:26.601452 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:26.603578 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:26.607196 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.607199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.607202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.607609 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.607967 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.607969 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.607971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.608088 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.608699 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:26.608702 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:26.608820 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 01:06:26.608922 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 01:06:26.609091 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 01:06:26.609548 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.596089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41048 (local address 127.5.108.189:34693)
0806 01:06:26.596329 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:26.596336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:26.596377 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:06:26.596445 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:26.596450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:26.596527 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:26.596635 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:26.597393 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.598477 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.599672 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.600420 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.600551 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.603772 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:26.603779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:26.603784 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:26.603822 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:26.607056 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.607717 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.607722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.607726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.607864 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.608198 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.608203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.608206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.608587 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:26.608747 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:26.609076 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 01:06:26.609283 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:06:26.610532 6387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:06:26.611518 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.611120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36408)
0806 01:06:26.611255 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.611320 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 01:06:26.612401 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.612046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36422)
0806 01:06:26.612205 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.612286 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 01:06:26.612632 6387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:40753: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
I20260806 01:06:26.657158 6492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } attempt: 3
I20260806 01:06:26.658314 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.657848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36430)
0806 01:06:26.658035 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.658138 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:06:26.659512 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.659049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36444)
0806 01:06:26.659276 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.659354 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260806 01:06:26.659778 6492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:40753: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
I20260806 01:06:26.663821 6387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } attempt: 1
I20260806 01:06:26.664806 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.664432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36446)
0806 01:06:26.664592 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.664665 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 01:06:26.665809 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.665520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36448)
0806 01:06:26.665659 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.665720 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 01:06:26.666112 6387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:40753: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
I20260806 01:06:26.734308 6387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } attempt: 2
I20260806 01:06:26.735555 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.735038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36454)
0806 01:06:26.735255 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.735339 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 01:06:26.736501 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.736173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36462)
0806 01:06:26.736322 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.736394 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 01:06:26.736750 6387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:40753: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
I20260806 01:06:26.802996 6492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } attempt: 4
I20260806 01:06:26.804294 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.803785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36466)
0806 01:06:26.804018 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.804124 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 01:06:26.805392 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.805034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36474)
0806 01:06:26.805212 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.805285 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 01:06:26.805632 6492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:40753: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
W20260806 01:06:26.806138 6502 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:06:26.806483 6502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:26.806589 6502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:26.806746 6502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:26.813949 6502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:26.814028 6502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:26.814141 6502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:26.814227 6502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:26.814317 6502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:26.823952 6502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:39399,127.5.108.189:34693,127.5.108.188:40753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab
--principal=kudu/127.5.108.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.188:40753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.108.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:26.825196 6502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:26.826502 6502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:26.837883 6512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:26.837942 6515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:26.837834 6513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:26.838624 6502 server_base.cc:1034] running on GCE node
I20260806 01:06:26.839449 6502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:26.840739 6502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:26.841952 6502 hybrid_clock.cc:648] HybridClock initialized: now 1785978386841876 us; error 48 us; skew 500 ppm
I20260806 01:06:26.842288 6502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:06:26.842935 6387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } attempt: 3
I20260806 01:06:26.844058 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.843579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36478)
0806 01:06:26.843809 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.843895 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 01:06:26.844997 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.844661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36482)
0806 01:06:26.844817 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.844899 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 01:06:26.845228 6387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:40753: Network error: Client connection negotiation failed: client connection to 127.5.108.188:40753: connect: Connection refused (error 111)
I20260806 01:06:26.845781 6502 init.cc:377] Logged in from keytab as kudu/127.5.108.188@KRBTEST.COM (short username kudu)
I20260806 01:06:26.847224 6502 webserver.cc:460] Webserver started at http://127.5.108.188:41595/ using document root <none> and password file <none>
I20260806 01:06:26.847958 6502 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:26.848047 6502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:26.848302 6502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:26.851085 6502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/data/instance:
uuid: "85aebd8b12fb41559dd27873fd878f2a"
format_stamp: "Formatted at 2026-08-06 01:06:26 on dist-test-slave-x4qh"
server_key: "2f7bad2749dd69cbd8158c37a0d9f2ec"
server_key_iv: "385472bbe59fec1be1a5372ab639ffdb"
server_key_version: "encryptionkey@0"
I20260806 01:06:26.851970 6502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/wal/instance:
uuid: "85aebd8b12fb41559dd27873fd878f2a"
format_stamp: "Formatted at 2026-08-06 01:06:26 on dist-test-slave-x4qh"
server_key: "2f7bad2749dd69cbd8158c37a0d9f2ec"
server_key_iv: "385472bbe59fec1be1a5372ab639ffdb"
server_key_version: "encryptionkey@0"
I20260806 01:06:26.856832 6502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 01:06:26.860059 6523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:26.861301 6502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:06:26.861428 6502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/wal
uuid: "85aebd8b12fb41559dd27873fd878f2a"
format_stamp: "Formatted at 2026-08-06 01:06:26 on dist-test-slave-x4qh"
server_key: "2f7bad2749dd69cbd8158c37a0d9f2ec"
server_key_iv: "385472bbe59fec1be1a5372ab639ffdb"
server_key_version: "encryptionkey@0"
I20260806 01:06:26.861588 6502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:26.884883 6502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:26.888588 6502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:26.888801 6502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:26.907476 6502 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.188:40753
I20260806 01:06:26.907574 6595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.188:40753 every 8 connection(s)
I20260806 01:06:26.908690 6502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
I20260806 01:06:26.913138 6596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:26.918970 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6502
I20260806 01:06:26.919152 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978022745104-5554-0/minicluster-data/master-2/wal/instance
I20260806 01:06:26.919684 5554 external_mini_cluster.cc:1428] Setting key 0551870d63f743e1f23fa61d8af3d8c6
I20260806 01:06:26.926383 6596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:06:26.942672 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.921425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36494 (local address 127.5.108.188:40753)
0806 01:06:26.922257 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:26.922273 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:26.922344 (+ 71us) server_negotiation.cc:407] Connection header received
0806 01:06:26.924413 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:26.924437 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:26.924791 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:26.925133 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:26.926571 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.928017 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.929594 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.930560 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.930707 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.935354 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:26.935363 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:26.935418 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:26.935457 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:26.938515 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.939119 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.939131 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.939152 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.939358 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.939837 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.939842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.939845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.940298 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:26.940584 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:26.941023 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 01:06:26.941407 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":341,"threads_started":1}
I20260806 01:06:26.944422 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.928746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.5.108.190:39399)
0806 01:06:26.929351 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:26.929359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:26.930225 (+ 866us) server_negotiation.cc:407] Connection header received
0806 01:06:26.930391 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:26.930395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:26.930451 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:26.930536 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:26.931718 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.932935 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.934561 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.934959 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.935049 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.938319 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:26.938327 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:26.938333 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:26.938374 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:26.941610 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.942434 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.942437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.942439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.942554 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.942976 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.942984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.942987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.943336 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:26.943528 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:26.943895 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 01:06:26.944106 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":281,"threads_started":1}
I20260806 01:06:26.944453 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.928754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:39399 (local address 127.0.0.1:42588)
0806 01:06:26.930082 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.930126 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:26.930281 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:26.930624 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:26.930633 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:26.931270 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:26.931560 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.931575 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.933084 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.933087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.934417 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.934434 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.935182 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.935193 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.935443 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.936075 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:26.936089 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:26.938125 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:26.941769 (+ 3644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.941778 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.941785 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.942292 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.942674 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.942679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.942683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.942868 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.943477 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:26.943494 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:26.943621 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 01:06:26.943765 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 01:06:26.943971 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"thread_start_us":982,"threads_started":1}
I20260806 01:06:26.945967 6596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
I20260806 01:06:26.960892 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.946221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.5.108.190:39399)
0806 01:06:26.946414 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:26.946419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:26.946440 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:06:26.946475 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:26.946498 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:26.946582 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:26.946686 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:26.948241 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.949245 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.950262 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.950631 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.950718 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.955401 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:26.955410 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:26.955416 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:26.955453 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:26.958285 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.959035 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.959038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.959040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.959160 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.959590 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.959595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.959600 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.959841 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:26.960117 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:26.960390 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 01:06:26.960597 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:06:26.961438 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.946767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:34693 (local address 127.0.0.1:41050)
0806 01:06:26.946932 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.946949 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:26.947034 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:26.947333 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:26.947339 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:26.947862 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:26.948041 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.948054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.950302 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.950305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.951538 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.951553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.952237 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.952244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.952485 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.952979 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:26.953005 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:26.955580 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:26.959157 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.959161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.959165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.959568 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.959924 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.959928 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.959932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.960067 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.960546 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:26.960550 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:26.960720 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 01:06:26.960867 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 01:06:26.961133 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 01:06:26.961576 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.946868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.5.108.189:34693)
0806 01:06:26.947026 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:26.947030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:26.947044 (+ 14us) server_negotiation.cc:407] Connection header received
0806 01:06:26.947148 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:26.947151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:26.947205 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:26.947276 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:26.948205 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.949229 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.951687 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.952079 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.952180 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.955853 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:26.955858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:26.955861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:26.955890 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:26.959027 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.959681 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.959686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.959689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.959817 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.960182 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.960186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.960189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.960433 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:26.960575 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:26.961009 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 01:06:26.961217 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:06:26.962653 6596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
Aug 06 01:06:26 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
I20260806 01:06:26.976900 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.963346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36502)
0806 01:06:26.963532 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:26.963544 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:26.963608 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:26.963962 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:26.963966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:26.964305 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:26.964493 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.964504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.965508 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.965513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.966835 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:26.966848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.967740 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.967746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:26.967991 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.968660 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:26.968684 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:26.971005 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:26.974364 (+ 3359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.974369 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.974372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.974842 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.975190 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:26.975195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:26.975199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:26.975362 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:26.975982 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:26.975985 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:26.976105 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 01:06:26.976293 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 01:06:26.976587 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 01:06:26.977164 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.963610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36502 (local address 127.5.108.188:40753)
0806 01:06:26.963761 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:26.963766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:26.963785 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:06:26.963815 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:26.963817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:26.963857 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:26.963947 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:26.964633 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.965323 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.966987 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.967599 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.967714 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.971198 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:26.971202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:26.971205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:26.971232 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:26.974234 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.974963 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.974967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.974969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.975070 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.975476 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.975479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.975481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.975858 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:26.976008 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:26.976373 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 01:06:26.976794 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":56}
I20260806 01:06:26.978276 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:26.962553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.5.108.189:34693)
0806 01:06:26.962699 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:26.962702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:26.962752 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:06:26.962965 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:26.962970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:26.963038 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:26.963126 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:26.964477 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.965306 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.966987 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:26.967586 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:26.967714 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:26.972580 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:26.972584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:26.972587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:26.972614 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:26.975470 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.976196 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.976201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.976204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.976363 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:26.976802 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:26.976806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:26.976809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:26.977289 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:26.977570 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:26.977876 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 01:06:26.978033 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 01:06:26.982769 6596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a: Bootstrap starting.
I20260806 01:06:26.983175 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:26.985975 6596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:26.987013 6596 log.cc:824] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:26.985878 6353 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7ca8328343ae4b21a126cef2e3c98b33"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42592
I20260806 01:06:26.989697 6596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a: No bootstrap required, opened a new log
I20260806 01:06:26.996680 6596 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } }
I20260806 01:06:26.997043 6596 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85aebd8b12fb41559dd27873fd878f2a, State: Initialized, Role: FOLLOWER
I20260806 01:06:26.997483 6596 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } }
I20260806 01:06:26.998342 6605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:26.998761 6605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:26.999213 6596 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:06:27.008388 6622 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:06:27.008533 6622 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:06:27.006326 6502 master_runner.cc:186] Error getting master registration for 127.5.108.190:39399: OK, instance_id { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" instance_seqno: 1785978385890020 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:06:27.017136 6502 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:06:27.022472 6387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } attempt: 4
Aug 06 01:06:27 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978385, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
I20260806 01:06:27.038333 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.023118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36516)
0806 01:06:27.023344 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:27.023386 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:27.023512 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:27.023923 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:27.023928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:27.024524 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:27.024744 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:27.024757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.026225 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.026231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:27.027692 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:27.027708 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.028637 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.028643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:27.028835 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.029619 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:27.029642 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:27.032459 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:27.035637 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:27.035642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:27.035646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:27.036124 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:27.036513 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:27.036518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:27.036522 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:27.036684 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:27.037373 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:27.037379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:27.037552 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 01:06:27.037689 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 01:06:27.037947 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 01:06:27.038347 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.023307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.5.108.188:40753)
0806 01:06:27.023573 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.023578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.023598 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:06:27.023678 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.023686 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.023757 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.023860 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.024896 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.026065 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.027832 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.028482 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.028595 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.032634 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:27.032660 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:27.032663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:27.032692 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:27.035498 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.036249 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.036254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.036257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.036393 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.036802 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.036807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.036810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.037231 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:27.037385 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.037812 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.038017 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260806 01:06:27.042747 6387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33: Bootstrap starting.
I20260806 01:06:27.046885 6387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:27.048360 6387 log.cc:824] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:27.051898 6387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33: No bootstrap required, opened a new log
I20260806 01:06:27.060883 6387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } }
I20260806 01:06:27.061370 6387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca8328343ae4b21a126cef2e3c98b33, State: Initialized, Role: FOLLOWER
I20260806 01:06:27.061995 6387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } }
I20260806 01:06:27.063303 6625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:27.064038 6625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:27.064666 6387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:27.064821 6353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 01:06:27.065012 6353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:27.068564 6353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } }
I20260806 01:06:27.070106 6353 leader_election.cc:288] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [CANDIDATE]: Term 1 election: Requested vote from peers 4765f5e824cc4c3295a732716e85af1d (127.5.108.189:34693), 85aebd8b12fb41559dd27873fd878f2a (127.5.108.188:40753)
I20260806 01:06:27.072114 6559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ca8328343ae4b21a126cef2e3c98b33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "85aebd8b12fb41559dd27873fd878f2a"
I20260806 01:06:27.072772 6559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:27.077198 6559 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca8328343ae4b21a126cef2e3c98b33 in term 1.
I20260806 01:06:27.078049 6309 leader_election.cc:302] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca8328343ae4b21a126cef2e3c98b33, 85aebd8b12fb41559dd27873fd878f2a; no voters:
I20260806 01:06:27.078648 6625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:27.079427 6625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [term 1 LEADER]: Becoming Leader. State: Replica: 7ca8328343ae4b21a126cef2e3c98b33, State: Running, Role: LEADER
I20260806 01:06:27.080001 6625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } }
I20260806 01:06:27.086712 6634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ca8328343ae4b21a126cef2e3c98b33. Latest consensus state: current_term: 1 leader_uuid: "7ca8328343ae4b21a126cef2e3c98b33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:27.087728 6634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:27.091943 6638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:27.103050 6638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:27.104866 6492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } attempt: 5
Aug 06 01:06:27 dist-test-slave-x4qh krb5kdc[6263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
I20260806 01:06:27.119755 6653 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:06:27.120540 6559 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [term 1 FOLLOWER]: Refusing update from remote peer 7ca8328343ae4b21a126cef2e3c98b33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:06:27.121573 6625 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:27.133256 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.105532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:40753 (local address 127.0.0.1:36520)
0806 01:06:27.105918 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:27.105939 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:27.106064 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:27.106956 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:27.106964 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:27.107513 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:27.107732 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:27.107745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.109193 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.109199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:27.110574 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:27.110590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.111772 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.111778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:27.112022 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.112753 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:27.112776 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:27.115628 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:27.130111 (+ 14483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:27.130117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:27.130121 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:27.130688 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:27.131263 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:27.131269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:27.131273 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:27.131484 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:27.132218 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:27.132223 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:27.132397 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 01:06:27.132539 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 01:06:27.132845 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253}
I20260806 01:06:27.134223 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.106300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36520 (local address 127.5.108.188:40753)
0806 01:06:27.106479 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.106484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.106503 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:06:27.106577 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.106584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.106655 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.106758 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.107905 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.109018 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.110744 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.111448 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.111566 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.115873 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:27.115880 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:27.115885 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:27.115924 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:27.129950 (+ 14026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.130835 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.130842 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.130850 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.131017 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.131628 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.131633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.131637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.131993 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:27.132137 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.133627 (+ 1490us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.133853 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 01:06:27.134508 6318 tablet.cc:2369] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:06:27.123724 6283 master_runner.cc:186] Error getting master registration for 127.5.108.189:34693: OK, instance_id { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" instance_seqno: 1785978386382051 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:06:27.139554 6283 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 01:06:27.140533 6529 tablet.cc:2369] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:06:27.142400 6605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ca8328343ae4b21a126cef2e3c98b33. Latest consensus state: current_term: 1 leader_uuid: "7ca8328343ae4b21a126cef2e3c98b33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:27.142768 6605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:27.143338 6492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d: Bootstrap starting.
I20260806 01:06:27.147758 6654 mvcc.cc:205] Tried to move back new op lower bound from 7315367473618972672 to 7315367473489580032. Current Snapshot: MvccSnapshot[applied={T|T < 7315367473618972672}]
I20260806 01:06:27.150668 6649 mvcc.cc:205] Tried to move back new op lower bound from 7315367473618972672 to 7315367473489580032. Current Snapshot: MvccSnapshot[applied={T|T < 7315367473618972672}]
I20260806 01:06:27.152120 6605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ca8328343ae4b21a126cef2e3c98b33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:27.152120 6492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:27.152482 6605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:27.158866 6492 log.cc:824] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:27.160152 6634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ca8328343ae4b21a126cef2e3c98b33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:27.160416 6634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:27.162302 6638 catalog_manager.cc:1347] Generated new cluster ID: 97cd7082bdeb4a8bb8211c734da01e15
I20260806 01:06:27.162418 6638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:27.189380 6492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d: No bootstrap required, opened a new log
I20260806 01:06:27.200268 6638 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:27.203014 6638 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:27.210870 6492 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } }
I20260806 01:06:27.211298 6492 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4765f5e824cc4c3295a732716e85af1d, State: Initialized, Role: FOLLOWER
I20260806 01:06:27.211858 6492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } }
I20260806 01:06:27.214564 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.168280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.5.108.188:40753)
0806 01:06:27.168494 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.168502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.183939 (+ 15437us) server_negotiation.cc:407] Connection header received
0806 01:06:27.184538 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.184545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.184631 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.184738 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.188803 (+ 4065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.189778 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.193216 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.193939 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.194024 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.201952 (+ 7928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:27.201958 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:27.201962 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:27.202003 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:27.206047 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.207579 (+ 1532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.207586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.207589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.207747 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.208588 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.208593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.208596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.208962 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:27.209240 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.210320 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.213724 (+ 3404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:06:27.216612 6638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33: Generated new TSK 0
I20260806 01:06:27.217626 6638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:27.222719 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.153275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.5.108.189:34693)
0806 01:06:27.153474 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.153479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.173011 (+ 19532us) server_negotiation.cc:407] Connection header received
0806 01:06:27.173295 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.173304 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.173408 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.173489 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.187956 (+ 14467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.189103 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.192045 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.192802 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.192917 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.213752 (+ 20835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:27.213760 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:27.213767 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:27.213814 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:27.219145 (+ 5331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.219960 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.219967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.219970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.220139 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.220836 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.220841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.220844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.221214 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:27.221447 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.222043 (+ 596us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.222305 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:06:27.224498 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.153973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42594 (local address 127.5.108.190:39399)
0806 01:06:27.154168 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.154173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.167699 (+ 13526us) server_negotiation.cc:407] Connection header received
0806 01:06:27.171712 (+ 4013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.171722 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.171846 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.171959 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.184566 (+ 12607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.185781 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.190622 (+ 4841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.191319 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.191457 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.208607 (+ 17150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:27.208615 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:27.208619 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:27.208675 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:27.212800 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.219990 (+ 7190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.219995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.219998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.220153 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:27.220631 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:27.220665 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:27.220671 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:27.221363 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:27.221920 (+ 557us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.223019 (+ 1099us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.223649 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 01:06:27.226464 6492 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:27.224318 6669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:27.227918 6669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:27.228844 6453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:27.232290 6453 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [term 1 FOLLOWER]: Refusing update from remote peer 7ca8328343ae4b21a126cef2e3c98b33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:06:27.233278 6634 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:27.233275 6454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ca8328343ae4b21a126cef2e3c98b33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4765f5e824cc4c3295a732716e85af1d"
I20260806 01:06:27.236796 6454 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca8328343ae4b21a126cef2e3c98b33 in term 1.
I20260806 01:06:27.295503 6669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ca8328343ae4b21a126cef2e3c98b33. Latest consensus state: current_term: 1 leader_uuid: "7ca8328343ae4b21a126cef2e3c98b33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:27.295540 6686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ca8328343ae4b21a126cef2e3c98b33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:27.295962 6669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:27.296020 6686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:27.318502 6691 catalog_manager.cc:1259] Loaded cluster ID: 97cd7082bdeb4a8bb8211c734da01e15
I20260806 01:06:27.318681 6691 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d: loading cluster ID for follower catalog manager: success
I20260806 01:06:27.322395 6691 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d: acquiring CA information for follower catalog manager: success
I20260806 01:06:27.324888 6691 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4765f5e824cc4c3295a732716e85af1d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 01:06:27.359833 5554 mini_kdc.cc:348] Time spent kdestroy: real 0.121s user 0.000s sys 0.030s
W20260806 01:06:27.419849 6498 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.404207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.5.108.189:34693)
0806 01:06:27.404455 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.404463 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.416114 (+ 11651us) server_negotiation.cc:407] Connection header received
0806 01:06:27.418219 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.418225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.418305 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.418377 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.419623 (+ 1246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41072: BlockingRecv error: recv got EOF from 127.0.0.1:41072 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 01:06:27.423817 6602 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.412500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.5.108.190:39399)
0806 01:06:27.412726 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.412733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.421411 (+ 8678us) server_negotiation.cc:407] Connection header received
0806 01:06:27.421562 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.421567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.421667 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.421792 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.423544 (+ 1752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42598: BlockingRecv error: recv got EOF from 127.0.0.1:42598 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 01:06:27.426129 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.416642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.5.108.188:40753)
0806 01:06:27.416853 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.416861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.422156 (+ 5295us) server_negotiation.cc:407] Connection header received
0806 01:06:27.424592 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.424598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.424680 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.424766 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.425876 (+ 1110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36534: BlockingRecv error: recv got EOF from 127.0.0.1:36534 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:06:27.482645 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.464178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41082 (local address 127.5.108.189:34693)
0806 01:06:27.464425 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.464431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.474975 (+ 10544us) server_negotiation.cc:407] Connection header received
0806 01:06:27.475534 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.475540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.475731 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.475858 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:27.476569 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.477649 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.479627 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.480245 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.480343 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.481134 (+ 791us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:27.481791 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:27.481912 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.482176 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.482269 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 01:06:27.529963 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.516567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36540 (local address 127.5.108.188:40753)
0806 01:06:27.516757 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.516764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.528139 (+ 11375us) server_negotiation.cc:407] Connection header received
0806 01:06:27.528468 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.528475 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.528609 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.528696 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.529728 (+ 1032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36540: BlockingRecv error: recv got EOF from 127.0.0.1:36540 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:06:27.594710 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.576508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.5.108.189:34693)
0806 01:06:27.576720 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.576728 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.587414 (+ 10686us) server_negotiation.cc:407] Connection header received
0806 01:06:27.587987 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.587996 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.588142 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.588225 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:27.588972 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.590148 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.592233 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.592984 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.593092 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.593787 (+ 695us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:27.593942 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:27.594010 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.594289 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.594382 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 01:06:27.650388 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.637374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36550 (local address 127.5.108.188:40753)
0806 01:06:27.637605 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.637611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.648251 (+ 10640us) server_negotiation.cc:407] Connection header received
0806 01:06:27.648794 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.648802 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.648938 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.649023 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.650148 (+ 1125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36550: BlockingRecv error: recv got EOF from 127.0.0.1:36550 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:06:27.705045 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.687514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.5.108.189:34693)
0806 01:06:27.687696 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.687702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.698255 (+ 10553us) server_negotiation.cc:407] Connection header received
0806 01:06:27.698713 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.698719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.698803 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.698867 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:27.699526 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.700673 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.702703 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.703399 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.703509 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.704141 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:27.704299 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:27.704397 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.704615 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.704716 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 01:06:27.750814 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.738069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36562 (local address 127.5.108.188:40753)
0806 01:06:27.738287 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.738294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.748857 (+ 10563us) server_negotiation.cc:407] Connection header received
0806 01:06:27.749337 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.749343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.749435 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.749505 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.750584 (+ 1079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36562: BlockingRecv error: recv got EOF from 127.0.0.1:36562 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:06:27.777328 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.771426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.5.108.189:34693)
0806 01:06:27.771650 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.771659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.772219 (+ 560us) server_negotiation.cc:407] Connection header received
0806 01:06:27.772480 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.772485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.772577 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.772692 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:27.773135 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.774068 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.775685 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.776119 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.776250 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.776638 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:27.776733 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:27.776828 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.776968 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.777040 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 01:06:27.795475 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.793598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36564 (local address 127.5.108.188:40753)
0806 01:06:27.793860 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.793867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.794053 (+ 186us) server_negotiation.cc:407] Connection header received
0806 01:06:27.794298 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.794304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.794389 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.794498 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.795238 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36564: BlockingRecv error: recv got EOF from 127.0.0.1:36564 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
I20260806 01:06:27.810350 6634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ca8328343ae4b21a126cef2e3c98b33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca8328343ae4b21a126cef2e3c98b33" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 39399 } } peers { permanent_uuid: "4765f5e824cc4c3295a732716e85af1d" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 34693 } } peers { permanent_uuid: "85aebd8b12fb41559dd27873fd878f2a" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 40753 } } }
I20260806 01:06:27.810765 6634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca8328343ae4b21a126cef2e3c98b33 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:27.817668 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.806903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.5.108.189:34693)
0806 01:06:27.807116 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.807125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.807701 (+ 576us) server_negotiation.cc:407] Connection header received
0806 01:06:27.807957 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.807962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.808026 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.808131 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:27.808548 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.809671 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.816000 (+ 6329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.816430 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.816573 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.816979 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:27.817072 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:27.817171 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.817310 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.817392 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 01:06:27.825506 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.823658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36572 (local address 127.5.108.188:40753)
0806 01:06:27.823842 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.823848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.824146 (+ 298us) server_negotiation.cc:407] Connection header received
0806 01:06:27.824254 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.824259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.824394 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.824544 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.825224 (+ 680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36572: BlockingRecv error: recv got EOF from 127.0.0.1:36572 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:06:27.857327 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.850355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.5.108.189:34693)
0806 01:06:27.850534 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.850540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.850882 (+ 342us) server_negotiation.cc:407] Connection header received
0806 01:06:27.851077 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.851081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.851164 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.851252 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:27.851916 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.853174 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.855176 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.855785 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.855907 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.856467 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:27.856640 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:27.856746 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.856960 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.857043 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 01:06:27.867316 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.864791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36574 (local address 127.5.108.188:40753)
0806 01:06:27.865007 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.865013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.865443 (+ 430us) server_negotiation.cc:407] Connection header received
0806 01:06:27.865674 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.865681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.865779 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.865881 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.867052 (+ 1171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36574: BlockingRecv error: recv got EOF from 127.0.0.1:36574 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:06:27.911039 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.904170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.5.108.189:34693)
0806 01:06:27.904479 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.904489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.905032 (+ 543us) server_negotiation.cc:407] Connection header received
0806 01:06:27.905299 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.905308 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.905379 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.905494 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:27.905923 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.906738 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.908884 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.909537 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.909638 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.910187 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:27.910334 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:27.910393 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.910632 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.910715 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 01:06:27.925359 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:27.923133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.5.108.188:40753)
0806 01:06:27.923308 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.923314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.923839 (+ 525us) server_negotiation.cc:407] Connection header received
0806 01:06:27.923965 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.923970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.924055 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.924161 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:27.925095 (+ 934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36590: BlockingRecv error: recv got EOF from 127.0.0.1:36590 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:06:27.997676 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:27.992834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.5.108.189:34693)
0806 01:06:27.993088 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:27.993095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:27.993405 (+ 310us) server_negotiation.cc:407] Connection header received
0806 01:06:27.993576 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:27.993580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:27.993662 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:27.993742 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:27.994147 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.994871 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.996166 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:27.996579 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:27.996670 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:27.997010 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:27.997110 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:27.997174 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:27.997303 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 01:06:27.997363 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 01:06:28.005357 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:28.003730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.5.108.188:40753)
0806 01:06:28.003926 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:28.003930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:28.004241 (+ 311us) server_negotiation.cc:407] Connection header received
0806 01:06:28.004408 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:28.004412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:28.004477 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:28.004556 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:28.005207 (+ 651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36592: BlockingRecv error: recv got EOF from 127.0.0.1:36592 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 01:06:28.010339 6622 catalog_manager.cc:1259] Loaded cluster ID: 97cd7082bdeb4a8bb8211c734da01e15
I20260806 01:06:28.010445 6622 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a: loading cluster ID for follower catalog manager: success
I20260806 01:06:28.012924 6622 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a: acquiring CA information for follower catalog manager: success
I20260806 01:06:28.014604 6622 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 85aebd8b12fb41559dd27873fd878f2a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:06:28.142616 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:28.137717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.5.108.189:34693)
0806 01:06:28.137994 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:28.138000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:28.138291 (+ 291us) server_negotiation.cc:407] Connection header received
0806 01:06:28.138466 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:28.138469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:28.138534 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:28.138609 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:28.138953 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.139759 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.141047 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.141518 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.141606 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:28.141952 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:28.142054 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:28.142116 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:28.142233 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 01:06:28.142297 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:06:28.153267 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:28.148053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36606 (local address 127.5.108.188:40753)
0806 01:06:28.148260 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:28.148266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:28.148508 (+ 242us) server_negotiation.cc:407] Connection header received
0806 01:06:28.148613 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:28.148616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:28.148767 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:28.148858 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:28.149208 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.149947 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.151175 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.151627 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.151715 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:28.152166 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:28.152697 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:28.152805 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:28.152878 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 01:06:28.152935 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:06:28.168864 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:28.159096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.5.108.189:34693)
0806 01:06:28.159247 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:28.159254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:28.162982 (+ 3728us) server_negotiation.cc:407] Connection header received
0806 01:06:28.163185 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:28.163190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:28.163268 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:28.163331 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:28.164156 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.165158 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.166914 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.167434 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.167539 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:28.168103 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:28.168250 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:28.168348 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:28.168493 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 01:06:28.168588 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 01:06:28.170200 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:28.159830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.5.108.188:40753)
0806 01:06:28.159983 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:28.159989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:28.164362 (+ 4373us) server_negotiation.cc:407] Connection header received
0806 01:06:28.164571 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:28.164576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:28.164650 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:28.164730 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:28.165357 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.166395 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.168176 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.168795 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.168925 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:28.169587 (+ 662us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:28.169673 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:28.169782 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:28.169897 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 01:06:28.169969 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:06:28.170810 6849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:28.159145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.5.108.190:39399)
0806 01:06:28.162612 (+ 3467us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:28.162619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:28.162639 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:06:28.162763 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:28.162768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:28.162991 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:28.163180 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:28.163948 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.165108 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.167257 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:28.167952 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:28.168063 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:28.168969 (+ 906us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:28.169440 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:28.169561 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:28.170444 (+ 883us) server_negotiation.cc:299] Negotiation successful
0806 01:06:28.170526 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3334,"thread_start_us":3219,"threads_started":1}
I20260806 01:06:28.175490 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6283
I20260806 01:06:28.176753 6378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:28.280726 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6283
W20260806 01:06:28.311015 6524 connection.cc:535] server connection from 127.0.0.1:36516 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 01:06:28.311800 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6396
I20260806 01:06:28.312951 6485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:28.416124 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6396
I20260806 01:06:28.442160 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6502
I20260806 01:06:28.443404 6588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:28.542204 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6502
2026-08-06T01:06:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4618 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:28 dist-test-slave-x4qh krb5kdc[6858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:28 dist-test-slave-x4qh krb5kdc[6858](info): set up 2 sockets
Aug 06 01:06:28 dist-test-slave-x4qh krb5kdc[6858](info): commencing operation
krb5kdc: starting...
W20260806 01:06:29.785043 5554 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/test-user.keytab.
Aug 06 01:06:29 dist-test-slave-x4qh krb5kdc[6858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.108.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
Entry for principal kudu/127.5.108.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab.
I20260806 01:06:30.072654 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:35167
--webserver_interface=127.5.108.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:35167,127.5.108.189:44863,127.5.108.188:33593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:30.395943 6877 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:06:30.396308 6877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:30.396437 6877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:30.396569 6877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:30.403332 6877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:30.403466 6877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:30.403599 6877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:30.403692 6877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:30.403810 6877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:30.412902 6877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:35167,127.5.108.189:44863,127.5.108.188:33593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.190.keytab
--principal=kudu/127.5.108.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.190:35167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.108.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:30.414238 6877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:30.415601 6877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:30.428252 6889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:30.428261 6885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:30.428539 6883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:30.429194 6877 server_base.cc:1034] running on GCE node
I20260806 01:06:30.429881 6877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:30.431071 6877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:30.432288 6877 hybrid_clock.cc:648] HybridClock initialized: now 1785978390432218 us; error 45 us; skew 500 ppm
I20260806 01:06:30.432605 6877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:30 dist-test-slave-x4qh krb5kdc[6858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978390, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:30.438596 6877 init.cc:377] Logged in from keytab as kudu/127.5.108.190@KRBTEST.COM (short username kudu)
I20260806 01:06:30.439975 6877 webserver.cc:460] Webserver started at http://127.5.108.190:38829/ using document root <none> and password file <none>
I20260806 01:06:30.440589 6877 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:30.440699 6877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:30.440984 6877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:30.443737 6877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/data/instance:
uuid: "b75874e0ca9b4472b03475baaba05ea9"
format_stamp: "Formatted at 2026-08-06 01:06:30 on dist-test-slave-x4qh"
server_key: "be50bc70b5c44cbefcb06300e350c412"
server_key_iv: "2e67e27f60272caea1d6ecb7c7167429"
server_key_version: "encryptionkey@0"
I20260806 01:06:30.444526 6877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/wal/instance:
uuid: "b75874e0ca9b4472b03475baaba05ea9"
format_stamp: "Formatted at 2026-08-06 01:06:30 on dist-test-slave-x4qh"
server_key: "be50bc70b5c44cbefcb06300e350c412"
server_key_iv: "2e67e27f60272caea1d6ecb7c7167429"
server_key_version: "encryptionkey@0"
I20260806 01:06:30.449061 6877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:06:30.452157 6901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:30.453415 6877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:06:30.453624 6877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/wal
uuid: "b75874e0ca9b4472b03475baaba05ea9"
format_stamp: "Formatted at 2026-08-06 01:06:30 on dist-test-slave-x4qh"
server_key: "be50bc70b5c44cbefcb06300e350c412"
server_key_iv: "2e67e27f60272caea1d6ecb7c7167429"
server_key_version: "encryptionkey@0"
I20260806 01:06:30.453889 6877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:30.489079 6877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:30.493014 6877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:30.493301 6877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:30.511790 6877 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.190:35167
I20260806 01:06:30.511818 6981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.190:35167 every 8 connection(s)
I20260806 01:06:30.513084 6877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:30.516594 6983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:30.520861 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6877
I20260806 01:06:30.521064 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-0/wal/instance
I20260806 01:06:30.521597 5554 external_mini_cluster.cc:1428] Setting key 947a965a9fee6694d69a492ac97aee38
I20260806 01:06:30.523628 6983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:06:30 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978390, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:06:30.548080 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.527550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:35167 (local address 127.0.0.1:40898)
0806 01:06:30.528354 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.528408 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:30.530924 (+ 2516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:30.531748 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:30.531757 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:30.532514 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:30.533036 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:30.533063 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:30.534551 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:30.534558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:30.536286 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:30.536303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:30.537147 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:30.537156 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:30.537347 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:30.538005 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:30.538026 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:30.540136 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:30.544465 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:30.544475 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:30.544482 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:30.544859 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:30.545333 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:30.545335 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:30.545338 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:30.545467 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:30.546425 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:30.546429 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:30.546607 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 01:06:30.546794 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 01:06:30.547103 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":333,"threads_started":1}
I20260806 01:06:30.548116 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.527550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40898 (local address 127.5.108.190:35167)
0806 01:06:30.529239 (+ 1689us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:30.529245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:30.529288 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:06:30.531253 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:30.531258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:30.531537 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:30.531680 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:30.533259 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:30.534407 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:30.536418 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:30.536928 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:30.537045 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:30.540366 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:30.540378 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:30.540394 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:30.540440 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:30.544314 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:30.545006 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:30.545016 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:30.545022 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:30.545223 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:30.545610 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:30.545617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:30.545621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:30.546292 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:30.546486 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:30.546984 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 01:06:30.547274 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1508,"spinlock_wait_cycles":22272,"thread_start_us":1288,"threads_started":1}
I20260806 01:06:30.550879 6983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:06:30.551949 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.551593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56756)
0806 01:06:30.551751 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.551856 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 01:06:30.552292 6906 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
I20260806 01:06:30.553850 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.553526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56770)
0806 01:06:30.553655 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.553717 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:06:30.554113 6983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:44863: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.108.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab.
Entry for principal kudu/127.5.108.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab.
I20260806 01:06:30.595831 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.189:44863
--webserver_interface=127.5.108.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab
--principal=kudu/127.5.108.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:35167,127.5.108.189:44863,127.5.108.188:33593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:06:30.603504 6983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } attempt: 1
I20260806 01:06:30.604692 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.604227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56786)
0806 01:06:30.604439 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.604555 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 01:06:30.605700 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.605383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56790)
0806 01:06:30.605516 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.605590 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:06:30.605933 6983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:44863: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
I20260806 01:06:30.674161 6983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } attempt: 2
I20260806 01:06:30.675304 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.674864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56806)
0806 01:06:30.675092 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.675178 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 01:06:30.676288 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.675954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56822)
0806 01:06:30.676093 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.676171 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:06:30.676517 6983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:44863: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
I20260806 01:06:30.767773 6983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } attempt: 3
I20260806 01:06:30.768862 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.768463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56830)
0806 01:06:30.768647 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.768732 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 01:06:30.769797 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.769489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56836)
0806 01:06:30.769604 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.769680 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 01:06:30.769994 6983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:44863: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
I20260806 01:06:30.913226 6983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } attempt: 4
I20260806 01:06:30.914433 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.913912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56846)
0806 01:06:30.914216 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.914300 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260806 01:06:30.915509 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:30.915133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56860)
0806 01:06:30.915249 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:30.915319 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 01:06:30.915752 6983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.189:44863: Network error: Client connection negotiation failed: client connection to 127.5.108.189:44863: connect: Connection refused (error 111)
W20260806 01:06:30.917789 6991 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:06:30.918107 6991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:30.918206 6991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:30.918331 6991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:30.925324 6991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:30.925422 6991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:30.925518 6991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:30.925611 6991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:30.925697 6991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:30.934391 6991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:35167,127.5.108.189:44863,127.5.108.188:33593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.189.keytab
--principal=kudu/127.5.108.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.189:44863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.108.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:30.935575 6991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:30.936810 6991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:30.947960 7003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:30.947903 7006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:30.947968 7001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:30.948716 6991 server_base.cc:1034] running on GCE node
I20260806 01:06:30.949486 6991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:30.950747 6991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:30.951979 6991 hybrid_clock.cc:648] HybridClock initialized: now 1785978390951922 us; error 51 us; skew 500 ppm
I20260806 01:06:30.952281 6991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:30 dist-test-slave-x4qh krb5kdc[6858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978390, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:30.955241 6991 init.cc:377] Logged in from keytab as kudu/127.5.108.189@KRBTEST.COM (short username kudu)
I20260806 01:06:30.956540 6991 webserver.cc:460] Webserver started at http://127.5.108.189:44533/ using document root <none> and password file <none>
I20260806 01:06:30.957222 6991 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:30.957351 6991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:30.957651 6991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:30.960554 6991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/data/instance:
uuid: "3fcd6bb12d4641d5ba2699478d242ff0"
format_stamp: "Formatted at 2026-08-06 01:06:30 on dist-test-slave-x4qh"
server_key: "738431618ed9b88ed838be4890b30e8d"
server_key_iv: "09e41b7c82d92e171e343b1f4bb64446"
server_key_version: "encryptionkey@0"
I20260806 01:06:30.961282 6991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/wal/instance:
uuid: "3fcd6bb12d4641d5ba2699478d242ff0"
format_stamp: "Formatted at 2026-08-06 01:06:30 on dist-test-slave-x4qh"
server_key: "738431618ed9b88ed838be4890b30e8d"
server_key_iv: "09e41b7c82d92e171e343b1f4bb64446"
server_key_version: "encryptionkey@0"
I20260806 01:06:30.966161 6991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:06:30.969179 7019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:30.970399 6991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:06:30.970579 6991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/wal
uuid: "3fcd6bb12d4641d5ba2699478d242ff0"
format_stamp: "Formatted at 2026-08-06 01:06:30 on dist-test-slave-x4qh"
server_key: "738431618ed9b88ed838be4890b30e8d"
server_key_iv: "09e41b7c82d92e171e343b1f4bb64446"
server_key_version: "encryptionkey@0"
I20260806 01:06:30.970791 6991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:30.987169 6991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:30.990729 6991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:30.991007 6991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:31.008797 6991 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.189:44863
I20260806 01:06:31.008831 7091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.189:44863 every 8 connection(s)
I20260806 01:06:31.010041 6991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/data/info.pb
I20260806 01:06:31.013298 7092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:31.013746 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 6991
I20260806 01:06:31.013962 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-1/wal/instance
I20260806 01:06:31.014402 5554 external_mini_cluster.cc:1428] Setting key 59ae1b4ba4f392a4f2129462ba9924a7
I20260806 01:06:31.019294 7092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978390, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:06:31.037537 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.021593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40912 (local address 127.5.108.190:35167)
0806 01:06:31.021771 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.021775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.022984 (+ 1209us) server_negotiation.cc:407] Connection header received
0806 01:06:31.024790 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.024795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.024885 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.024994 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.026367 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.027064 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.028519 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.028907 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.029033 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.032174 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.032178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.032182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.032212 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.034788 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.035464 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.035468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.035471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.035584 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.036000 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.036004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.036009 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.036270 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.036420 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.037057 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.037273 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:06:31.038288 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.022108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:35167 (local address 127.0.0.1:40912)
0806 01:06:31.022826 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.022861 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:31.024535 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:31.025233 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:31.025242 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:31.025716 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:31.026201 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.026217 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.027226 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.027230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.028367 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.028379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.029120 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.029127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.029330 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.029922 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:31.029942 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:31.031950 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:31.034916 (+ 2966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.034923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.034936 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.035301 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.035697 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.035703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.035706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.035887 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.036401 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:31.036408 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:31.036677 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 01:06:31.036927 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 01:06:31.037318 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":303,"threads_started":1}
I20260806 01:06:31.039956 7092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.108.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.108.188@KRBTEST.COM" created.
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978390, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
I20260806 01:06:31.054862 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.040739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56874)
0806 01:06:31.040911 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.040932 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:31.041023 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:31.042026 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:31.042030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:31.042432 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:31.042571 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.042578 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.044016 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.044024 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.044829 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.044840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.045584 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.045589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.045841 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.046347 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:31.046361 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:31.049349 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:31.052364 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.052369 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.052372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.052745 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.053124 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.053130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.053137 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.053277 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.053872 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:31.053877 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:31.054005 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 01:06:31.054156 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 01:06:31.054410 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":42}
I20260806 01:06:31.054919 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.041010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.5.108.189:44863)
0806 01:06:31.041678 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.041683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.041705 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:06:31.041750 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.041755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.041889 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.042036 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.042772 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.043885 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.044969 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.045445 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.045578 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.049555 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.049566 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.049576 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.049602 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.052154 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.052881 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.052889 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.052891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.053013 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.053381 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.053388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.053391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.053707 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.053944 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.054269 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.054516 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":520,"thread_start_us":374,"threads_started":1}
I20260806 01:06:31.057538 7092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:06:31.058884 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.058321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47902)
0806 01:06:31.058554 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.058691 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 01:06:31.059172 7020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
I20260806 01:06:31.060751 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.060361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47914)
0806 01:06:31.060520 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.060610 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:06:31.060961 7092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:33593: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.108.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab.
Entry for principal kudu/127.5.108.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab.
I20260806 01:06:31.082440 5554 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
/tmp/dist-test-taskq1gvHq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.188:33593
--webserver_interface=127.5.108.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab
--principal=kudu/127.5.108.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.108.148:38141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.108.190:35167,127.5.108.189:44863,127.5.108.188:33593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:06:31.110253 7092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } attempt: 1
I20260806 01:06:31.111321 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.110919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47916)
0806 01:06:31.111094 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.111188 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 01:06:31.112310 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.111996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47930)
0806 01:06:31.112149 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.112215 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:06:31.112534 7092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:33593: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
I20260806 01:06:31.180720 7092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } attempt: 2
I20260806 01:06:31.181852 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.181423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47936)
0806 01:06:31.181610 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.181697 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 01:06:31.182817 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.182458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47940)
0806 01:06:31.182607 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.182683 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 01:06:31.183032 7092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:33593: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
I20260806 01:06:31.215001 6983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } attempt: 5
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978390, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
I20260806 01:06:31.228459 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.215811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56880 (local address 127.5.108.189:44863)
0806 01:06:31.216029 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.216034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.216055 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:06:31.216191 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.216197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.216294 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.216404 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.217178 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.217855 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.218987 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.219514 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.219645 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.223172 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.223177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.223180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.223210 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.225859 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.226681 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.226685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.226687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.226793 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.227221 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.227224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.227226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.227529 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.227634 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.228111 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.228268 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 01:06:31.228945 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.215700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56880)
0806 01:06:31.215901 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.215924 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:31.216056 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:31.216457 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:31.216461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:31.216857 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:31.217032 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.217041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.217993 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.218000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.218842 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.218853 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.219661 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.219667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.219888 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.220545 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:31.220562 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:31.222956 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:31.225999 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.226004 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.226009 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.226558 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.226897 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.226903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.226909 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.227093 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.227655 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:31.227662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:31.227852 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 01:06:31.228040 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 01:06:31.228411 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 01:06:31.229913 6983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:06:31.230917 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.230594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47950)
0806 01:06:31.230738 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.230804 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 01:06:31.231866 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.231554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47966)
0806 01:06:31.231695 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.231777 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 01:06:31.232085 6983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:33593: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
I20260806 01:06:31.274294 7092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } attempt: 3
I20260806 01:06:31.275650 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.275104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47976)
0806 01:06:31.275316 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.275471 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 01:06:31.276651 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.276255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47992)
0806 01:06:31.276469 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.276548 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260806 01:06:31.276850 7092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:33593: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
I20260806 01:06:31.283293 6983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } attempt: 1
I20260806 01:06:31.284442 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.284012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:47996)
0806 01:06:31.284189 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.284266 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 01:06:31.285509 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.285185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48002)
0806 01:06:31.285338 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.285405 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:06:31.285758 6983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:33593: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
I20260806 01:06:31.354025 6983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } attempt: 2
I20260806 01:06:31.355216 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.354716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48014)
0806 01:06:31.354964 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.355046 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 01:06:31.356264 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.355923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48022)
0806 01:06:31.356093 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.356164 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 01:06:31.356514 6983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:33593: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
W20260806 01:06:31.416111 7104 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:06:31.416409 7104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:31.416519 7104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:31.416632 7104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 01:06:31.420069 7092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } attempt: 4
I20260806 01:06:31.421169 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.420699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48034)
0806 01:06:31.420932 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.421013 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:06:31.422084 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.421784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48036)
0806 01:06:31.421924 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.421992 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:06:31.422286 7092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:33593: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
W20260806 01:06:31.423975 7104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:31.424084 7104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:31.424194 7104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:31.424279 7104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:31.424423 7104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:31.433164 7104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.108.148:38141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.108.190:35167,127.5.108.189:44863,127.5.108.188:33593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/krb5kdc/kudu_127.5.108.188.keytab
--principal=kudu/127.5.108.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.108.188:33593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.108.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:31.434324 7104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:31.435621 7104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:31.447273 7117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:31.447538 7113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:31.447670 7112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:31.448464 7104 server_base.cc:1034] running on GCE node
I20260806 01:06:31.449131 7104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:31.450399 7104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:31.451637 7104 hybrid_clock.cc:648] HybridClock initialized: now 1785978391451573 us; error 43 us; skew 500 ppm
I20260806 01:06:31.451941 7104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:31.455010 7104 init.cc:377] Logged in from keytab as kudu/127.5.108.188@KRBTEST.COM (short username kudu)
I20260806 01:06:31.456312 7104 webserver.cc:460] Webserver started at http://127.5.108.188:43309/ using document root <none> and password file <none>
I20260806 01:06:31.456898 7104 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:31.456980 7104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:31.457221 7104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:31.460150 7104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/data/instance:
uuid: "b887fc3c1807458cbcbc6d7f6f01d324"
format_stamp: "Formatted at 2026-08-06 01:06:31 on dist-test-slave-x4qh"
server_key: "ae4aa2df3af8845f14aa83efb5288889"
server_key_iv: "c29b1edfc2d074bf587ce4817733bffd"
server_key_version: "encryptionkey@0"
I20260806 01:06:31.460906 7104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/wal/instance:
uuid: "b887fc3c1807458cbcbc6d7f6f01d324"
format_stamp: "Formatted at 2026-08-06 01:06:31 on dist-test-slave-x4qh"
server_key: "ae4aa2df3af8845f14aa83efb5288889"
server_key_iv: "c29b1edfc2d074bf587ce4817733bffd"
server_key_version: "encryptionkey@0"
I20260806 01:06:31.462787 6983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } attempt: 3
I20260806 01:06:31.463814 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.463331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48038)
0806 01:06:31.463575 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.463657 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260806 01:06:31.465021 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.464534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48044)
0806 01:06:31.464746 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.464866 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 01:06:31.465309 6983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.108.188:33593: Network error: Client connection negotiation failed: client connection to 127.5.108.188:33593: connect: Connection refused (error 111)
I20260806 01:06:31.466189 7104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:06:31.469503 7127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:31.470734 7104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:06:31.470923 7104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/data,/tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/wal
uuid: "b887fc3c1807458cbcbc6d7f6f01d324"
format_stamp: "Formatted at 2026-08-06 01:06:31 on dist-test-slave-x4qh"
server_key: "ae4aa2df3af8845f14aa83efb5288889"
server_key_iv: "c29b1edfc2d074bf587ce4817733bffd"
server_key_version: "encryptionkey@0"
I20260806 01:06:31.471123 7104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:31.484822 7104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:31.488682 7104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:31.488941 7104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:31.507828 7104 rpc_server.cc:307] RPC server started. Bound to: 127.5.108.188:33593
I20260806 01:06:31.507869 7204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.108.188:33593 every 8 connection(s)
I20260806 01:06:31.509675 7104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/data/info.pb
I20260806 01:06:31.514398 7206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:31.518671 5554 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu as pid 7104
I20260806 01:06:31.518899 5554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq1gvHq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978022745104-5554-0/minicluster-data/master-2/wal/instance
I20260806 01:06:31.519351 5554 external_mini_cluster.cc:1428] Setting key 846088f510d2ae753e80a9c59f02a2a3
I20260806 01:06:31.524241 7206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:06:31.541638 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.526686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.5.108.190:35167)
0806 01:06:31.526879 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.526883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.527735 (+ 852us) server_negotiation.cc:407] Connection header received
0806 01:06:31.527958 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.527963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.528026 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.528134 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.529231 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.529973 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.531296 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.532002 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.532166 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.535986 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.535993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.535998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.536037 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.538703 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.539525 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.539532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.539536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.539703 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.540119 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.540125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.540128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.540387 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.540581 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.541007 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.541232 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:06:31.541770 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.521182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48048 (local address 127.5.108.188:33593)
0806 01:06:31.521945 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.521955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.521995 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:06:31.523720 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.523742 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.524099 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.524496 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.526005 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.527992 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.529245 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.529841 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.530023 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.534556 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.534563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.534574 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.534602 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.537741 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.538480 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.538487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.538492 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.538638 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.539044 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.539049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.539052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.539640 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.539949 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.540363 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.540775 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":358,"threads_started":1}
I20260806 01:06:31.541796 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.526817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.190:35167 (local address 127.0.0.1:40922)
0806 01:06:31.527592 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.527637 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:31.527814 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:31.528204 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:31.528210 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:31.528801 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:31.529062 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.529071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.530155 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.530159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.531144 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.531157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.532371 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.532389 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.532740 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.533425 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:31.533448 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:31.535772 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:31.538855 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.538867 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.538874 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.539349 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.539827 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.539833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.539837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.539995 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.540543 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:31.540552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:31.540714 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 01:06:31.540884 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 01:06:31.541225 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":394,"threads_started":1}
I20260806 01:06:31.543579 7206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.190@KRBTEST.COM
I20260806 01:06:31.557852 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.544463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.189:44863 (local address 127.0.0.1:56890)
0806 01:06:31.544720 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.544734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:31.544859 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:31.545265 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:31.545270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:31.545663 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:31.545809 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.545817 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.547266 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.547270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.548303 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.548319 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.549086 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.549090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.549218 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.549759 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:31.549776 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:31.552334 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:31.555497 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.555501 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.555503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.555836 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.556227 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.556231 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.556235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.556367 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.556889 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:31.556893 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:31.557103 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 01:06:31.557260 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 01:06:31.557520 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260806 01:06:31.558156 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.544317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56890 (local address 127.5.108.189:44863)
0806 01:06:31.544596 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.544603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.544812 (+ 209us) server_negotiation.cc:407] Connection header received
0806 01:06:31.544995 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.545001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.545132 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.545240 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.545976 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.547093 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.548471 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.548907 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.549077 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.552517 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.552522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.552525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.552556 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.555318 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.555955 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.555962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.555965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.556110 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.556479 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.556484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.556487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.556768 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.556923 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.557352 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.557648 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260806 01:06:31.559144 7206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:06:31.561110 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.545265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.5.108.190:35167)
0806 01:06:31.546091 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.546097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.546351 (+ 254us) server_negotiation.cc:407] Connection header received
0806 01:06:31.546494 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.546497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.546548 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.546621 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.548012 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.548877 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.550200 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.550910 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.551030 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.555327 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.555333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.555337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.555401 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.558396 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.559122 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.559126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.559128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.559245 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.559668 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.559673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.559676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.559938 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.560216 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.560549 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.560788 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704}
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.188@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.108.189@KRBTEST.COM
I20260806 01:06:31.572223 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.560060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48056 (local address 127.5.108.188:33593)
0806 01:06:31.560196 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.560200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.560215 (+ 15us) server_negotiation.cc:407] Connection header received
0806 01:06:31.560258 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.560262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.560324 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.560424 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.561204 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.562046 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.563667 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.564077 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.564212 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.567148 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.567153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.567157 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.567193 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.569786 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.570438 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.570444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.570452 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.570552 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.570914 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.570918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.570921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.571155 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.571315 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.571656 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.571873 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":45}
I20260806 01:06:31.572397 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.559895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48056)
0806 01:06:31.560054 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.560070 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:31.560160 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:31.560466 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:31.560471 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:31.560873 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:31.561043 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.561053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.562196 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.562202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.563523 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.563535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.564219 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.564224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.564522 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.565063 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:31.565079 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:31.566973 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:31.569919 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.569926 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.569930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.570315 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.570665 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.570672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.570675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.570805 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.571286 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:31.571290 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:31.571438 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 01:06:31.571582 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 01:06:31.572017 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":161}
I20260806 01:06:31.576924 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.562781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56902 (local address 127.5.108.189:44863)
0806 01:06:31.562937 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.562942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.562960 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:06:31.563061 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.563066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.563126 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.563222 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.564419 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.565275 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.566740 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.567131 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.567283 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.571097 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.571101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.571105 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.571132 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.574130 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.574722 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.574727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.574733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.574902 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.575360 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.575366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.575369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.575696 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.576027 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.576299 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.576516 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:06:31.577781 7206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324: Bootstrap starting.
I20260806 01:06:31.581007 7206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:31.581303 5554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:06:31.582010 7206 log.cc:824] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:31.584769 7206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324: No bootstrap required, opened a new log
I20260806 01:06:31.583971 6942 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b75874e0ca9b4472b03475baaba05ea9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40926
I20260806 01:06:31.594579 7206 raft_consensus.cc:348] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } }
I20260806 01:06:31.594967 7206 raft_consensus.cc:729] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b887fc3c1807458cbcbc6d7f6f01d324, State: Initialized, Role: FOLLOWER
I20260806 01:06:31.595439 7206 consensus_queue.cc:260] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } }
I20260806 01:06:31.596309 7213 sys_catalog.cc:455] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.596742 7213 sys_catalog.cc:458] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:31.597301 7206 sys_catalog.cc:564] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:06:31.606352 7229 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:06:31.606489 7229 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:06:31.604645 7104 master_runner.cc:186] Error getting master registration for 127.5.108.190:35167: OK, instance_id { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" instance_seqno: 1785978390506693 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:06:31.615231 7104 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:06:31.642593 6983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } attempt: 4
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978390, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.190@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
I20260806 01:06:31.659332 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.643340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48068)
0806 01:06:31.643570 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.643592 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:31.643727 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:31.644146 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:31.644151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:31.644699 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:31.644922 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.644933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.646446 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.646451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.647929 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.647943 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.648925 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.648932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.649125 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.649811 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:31.649832 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:31.652538 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:31.656518 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.656523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.656528 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.656955 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.657333 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.657338 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.657341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.657475 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.658318 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:31.658325 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:31.658508 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 01:06:31.658648 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 01:06:31.658922 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 01:06:31.659403 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.643444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.5.108.188:33593)
0806 01:06:31.643647 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.643654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.643677 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:06:31.643868 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.643874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.643989 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.644100 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.645091 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.646290 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.648081 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.648778 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.648899 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.652713 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.652744 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.652749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.652807 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.656373 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.657097 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.657101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.657103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.657218 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.657598 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.657602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.657604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.658183 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.658341 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.658799 (+ 458us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.659014 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:06:31.665484 6983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9: Bootstrap starting.
I20260806 01:06:31.669294 6983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:31.670534 6983 log.cc:824] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:31.673791 6983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9: No bootstrap required, opened a new log
I20260806 01:06:31.681022 6983 raft_consensus.cc:348] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } }
I20260806 01:06:31.681391 6983 raft_consensus.cc:729] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b75874e0ca9b4472b03475baaba05ea9, State: Initialized, Role: FOLLOWER
I20260806 01:06:31.681834 6983 consensus_queue.cc:260] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } }
I20260806 01:06:31.682842 7234 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.683208 7234 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:31.684033 6983 sys_catalog.cc:564] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:31.684163 6942 raft_consensus.cc:482] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 01:06:31.684404 6942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:31.687817 6942 raft_consensus.cc:504] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } }
I20260806 01:06:31.689993 7166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b75874e0ca9b4472b03475baaba05ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b887fc3c1807458cbcbc6d7f6f01d324"
I20260806 01:06:31.690430 7166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:31.692790 7166 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b75874e0ca9b4472b03475baaba05ea9 in term 1.
I20260806 01:06:31.693437 6902 leader_election.cc:302] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b75874e0ca9b4472b03475baaba05ea9, b887fc3c1807458cbcbc6d7f6f01d324; no voters:
I20260806 01:06:31.693612 6942 leader_election.cc:288] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcd6bb12d4641d5ba2699478d242ff0 (127.5.108.189:44863), b887fc3c1807458cbcbc6d7f6f01d324 (127.5.108.188:33593)
I20260806 01:06:31.693938 7234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:31.694860 7234 raft_consensus.cc:686] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [term 1 LEADER]: Becoming Leader. State: Replica: b75874e0ca9b4472b03475baaba05ea9, State: Running, Role: LEADER
I20260806 01:06:31.695351 7234 consensus_queue.cc:237] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } }
I20260806 01:06:31.698597 7249 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b75874e0ca9b4472b03475baaba05ea9. Latest consensus state: current_term: 1 leader_uuid: "b75874e0ca9b4472b03475baaba05ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.699007 7249 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:31.709637 7253 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:06:31.710534 7252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:31.721463 7092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } attempt: 5
I20260806 01:06:31.730350 7252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 01:06:31 dist-test-slave-x4qh krb5kdc[6858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978390, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.108.189@KRBTEST.COM for kudu/127.5.108.188@KRBTEST.COM
W20260806 01:06:31.740770 7137 tablet.cc:2369] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:06:31.741397 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.721983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.108.188:33593 (local address 127.0.0.1:48072)
0806 01:06:31.722327 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:31.722348 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:31.722455 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:31.722989 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:31.722995 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:31.723541 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:31.723752 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.723765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.725347 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.725353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.726762 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:31.726779 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.727958 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.727967 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:31.728174 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.728922 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:31.728943 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:31.731790 (+ 2847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:31.736204 (+ 4414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.736217 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.736221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.737002 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.737685 (+ 683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:31.737711 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:31.737714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:31.737948 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:31.739042 (+ 1094us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:31.739048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:31.739303 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 01:06:31.739535 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 01:06:31.740105 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221}
I20260806 01:06:31.743993 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.722319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.5.108.188:33593)
0806 01:06:31.722496 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.722501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.722523 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:06:31.722576 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.722582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.722659 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.722783 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.723951 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.725152 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.726944 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.727612 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.727730 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.732040 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.732047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.732051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.732088 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.736001 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.737243 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.737251 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.737255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.737526 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.738217 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.738223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.738226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.738831 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.740185 (+ 1354us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.740697 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.742192 (+ 1495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 01:06:31.747623 6909 tablet.cc:2369] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:06:31.750916 7092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0: Bootstrap starting.
I20260806 01:06:31.754702 7092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:31.755965 7092 log.cc:824] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:31.714114 6877 master_runner.cc:186] Error getting master registration for 127.5.108.189:44863: OK, instance_id { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" instance_seqno: 1785978391003943 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:06:31.757545 6877 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:06:31.769312 7092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0: No bootstrap required, opened a new log
I20260806 01:06:31.774437 7166 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [term 1 FOLLOWER]: Refusing update from remote peer b75874e0ca9b4472b03475baaba05ea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:06:31.775424 7234 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:31.798626 7213 sys_catalog.cc:455] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b75874e0ca9b4472b03475baaba05ea9. Latest consensus state: current_term: 1 leader_uuid: "b75874e0ca9b4472b03475baaba05ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.799011 7213 sys_catalog.cc:458] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:31.814685 7213 sys_catalog.cc:455] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b75874e0ca9b4472b03475baaba05ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.814850 7270 mvcc.cc:205] Tried to move back new op lower bound from 7315367492693323776 to 7315367492388519936. Current Snapshot: MvccSnapshot[applied={T|T < 7315367492693323776}]
I20260806 01:06:31.814975 7213 sys_catalog.cc:458] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:31.815528 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.776331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.5.108.188:33593)
0806 01:06:31.776519 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.776524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.787976 (+ 11452us) server_negotiation.cc:407] Connection header received
0806 01:06:31.788279 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.788287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.788376 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.788452 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.790713 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.791787 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.803472 (+ 11685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.804131 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.804247 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.807667 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.807674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.807679 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.807721 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.811428 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.812298 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.812304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.812307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.812447 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.812954 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.812959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.812964 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.813456 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.813800 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.814385 (+ 585us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.814639 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:06:31.818395 7092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } }
I20260806 01:06:31.818822 7092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcd6bb12d4641d5ba2699478d242ff0, State: Initialized, Role: FOLLOWER
I20260806 01:06:31.819319 7092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } }
I20260806 01:06:31.819643 7258 mvcc.cc:205] Tried to move back new op lower bound from 7315367492693323776 to 7315367492388519936. Current Snapshot: MvccSnapshot[applied={T|T < 7315367492693323776}]
I20260806 01:06:31.827793 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.776160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40942 (local address 127.5.108.190:35167)
0806 01:06:31.776441 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.776447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.788708 (+ 12261us) server_negotiation.cc:407] Connection header received
0806 01:06:31.789152 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.789159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.789237 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.789309 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.792899 (+ 3590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.794016 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.799291 (+ 5275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.800054 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.800152 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.819913 (+ 19761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.819920 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.819925 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.819980 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.823842 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.824689 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.824695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.824699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.824898 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.825343 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.825348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.825351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.825749 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.826001 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.827162 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.827427 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260806 01:06:31.831656 7275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.833849 7275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:31.833707 7092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:31.834158 7051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:31.838580 7051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [term 1 FOLLOWER]: Refusing update from remote peer b75874e0ca9b4472b03475baaba05ea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:06:31.839313 7052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b75874e0ca9b4472b03475baaba05ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3fcd6bb12d4641d5ba2699478d242ff0"
I20260806 01:06:31.843201 7052 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b75874e0ca9b4472b03475baaba05ea9 in term 1.
I20260806 01:06:31.843875 7249 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:31.848675 7249 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b75874e0ca9b4472b03475baaba05ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.848922 7249 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:31.851259 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:31.790526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56912 (local address 127.5.108.189:44863)
0806 01:06:31.790697 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:31.790702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:31.812530 (+ 21828us) server_negotiation.cc:407] Connection header received
0806 01:06:31.813098 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:31.813104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:31.813208 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:31.813283 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:31.815460 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.816543 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.830773 (+ 14230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:31.831539 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:31.831650 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:31.838281 (+ 6631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:31.838288 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:31.838296 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:31.838333 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:31.842480 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.844173 (+ 1693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.844178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.844181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.844362 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:31.849125 (+ 4763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:31.849158 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:31.849161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:31.849588 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:31.849807 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:31.850417 (+ 610us) server_negotiation.cc:299] Negotiation successful
0806 01:06:31.850741 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:06:31.852861 7252 catalog_manager.cc:1347] Generated new cluster ID: af286bdd26f34ce99515a33b94359aa9
I20260806 01:06:31.852964 7252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:31.872371 7252 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:31.874766 7252 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:31.902340 7234 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b75874e0ca9b4472b03475baaba05ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.902653 7234 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:31.904021 7252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b75874e0ca9b4472b03475baaba05ea9: Generated new TSK 0
I20260806 01:06:31.904181 7275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b75874e0ca9b4472b03475baaba05ea9. Latest consensus state: current_term: 1 leader_uuid: "b75874e0ca9b4472b03475baaba05ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.904626 7275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:31.905118 7252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:06:31.905292 7287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b75874e0ca9b4472b03475baaba05ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75874e0ca9b4472b03475baaba05ea9" member_type: VOTER last_known_addr { host: "127.5.108.190" port: 35167 } } peers { permanent_uuid: "3fcd6bb12d4641d5ba2699478d242ff0" member_type: VOTER last_known_addr { host: "127.5.108.189" port: 44863 } } peers { permanent_uuid: "b887fc3c1807458cbcbc6d7f6f01d324" member_type: VOTER last_known_addr { host: "127.5.108.188" port: 33593 } } }
I20260806 01:06:31.905789 7287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:06:31.974455 7302 catalog_manager.cc:1259] Loaded cluster ID: af286bdd26f34ce99515a33b94359aa9
I20260806 01:06:31.974617 7302 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0: loading cluster ID for follower catalog manager: success
I20260806 01:06:31.985051 7302 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0: acquiring CA information for follower catalog manager: success
I20260806 01:06:31.987912 7302 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3fcd6bb12d4641d5ba2699478d242ff0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 01:06:32.056119 7100 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.045745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.5.108.189:44863)
0806 01:06:32.045944 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.045952 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.053202 (+ 7250us) server_negotiation.cc:407] Connection header received
0806 01:06:32.054482 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.054489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.054596 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.054691 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.055896 (+ 1205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56916: BlockingRecv error: recv got EOF from 127.0.0.1:56916 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 01:06:32.061033 7209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.047897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.5.108.188:33593)
0806 01:06:32.048115 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.048124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.057975 (+ 9851us) server_negotiation.cc:407] Connection header received
0806 01:06:32.059535 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.059542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.059634 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.059701 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.060801 (+ 1100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48096: BlockingRecv error: recv got EOF from 127.0.0.1:48096 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 01:06:32.062254 6987 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.049562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.5.108.190:35167)
0806 01:06:32.049780 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.049787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.057527 (+ 7740us) server_negotiation.cc:407] Connection header received
0806 01:06:32.058426 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.058431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.058532 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.058615 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.062027 (+ 3412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40956: BlockingRecv error: recv got EOF from 127.0.0.1:40956 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:06:32.107595 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.091886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.5.108.189:44863)
0806 01:06:32.092070 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.092079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.099451 (+ 7372us) server_negotiation.cc:407] Connection header received
0806 01:06:32.099981 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.099989 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.100274 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.100428 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.101209 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.102306 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.104465 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.105163 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.105284 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.106056 (+ 772us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.106769 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.106884 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.107124 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.107231 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 01:06:32.153846 7209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.140873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48098 (local address 127.5.108.188:33593)
0806 01:06:32.141063 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.141069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.151666 (+ 10597us) server_negotiation.cc:407] Connection header received
0806 01:06:32.152198 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.152206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.152349 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.152427 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.153548 (+ 1121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48098: BlockingRecv error: recv got EOF from 127.0.0.1:48098 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:06:32.214704 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.195440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56944 (local address 127.5.108.189:44863)
0806 01:06:32.195674 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.195683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.207999 (+ 12316us) server_negotiation.cc:407] Connection header received
0806 01:06:32.208258 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.208264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.208380 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.208471 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.209151 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.210328 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.212326 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.213024 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.213128 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.213779 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.213975 (+ 196us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.214038 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.214273 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.214373 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 01:06:32.269812 7209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.257320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.5.108.188:33593)
0806 01:06:32.257523 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.257528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.268338 (+ 10810us) server_negotiation.cc:407] Connection header received
0806 01:06:32.268541 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.268547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.268639 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.268753 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.269585 (+ 832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48112: BlockingRecv error: recv got EOF from 127.0.0.1:48112 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:06:32.288708 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.281390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56950 (local address 127.5.108.189:44863)
0806 01:06:32.281558 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.281563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.282066 (+ 503us) server_negotiation.cc:407] Connection header received
0806 01:06:32.282395 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.282402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.282497 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.282564 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.283278 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.284477 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.286552 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.287198 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.287345 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.287874 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.288026 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.288123 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.288297 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.288375 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 01:06:32.298425 7209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.296001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.5.108.188:33593)
0806 01:06:32.296161 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.296166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.296722 (+ 556us) server_negotiation.cc:407] Connection header received
0806 01:06:32.296998 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.297004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.297092 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.297160 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.298192 (+ 1032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48120: BlockingRecv error: recv got EOF from 127.0.0.1:48120 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:06:32.313910 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.306523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.5.108.189:44863)
0806 01:06:32.306771 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.306777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.307291 (+ 514us) server_negotiation.cc:407] Connection header received
0806 01:06:32.307597 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.307603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.307698 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.307757 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.308351 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.309505 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.311467 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.312088 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.312363 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.313036 (+ 673us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.313183 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.313257 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.313501 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.313609 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
W20260806 01:06:32.323369 7209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.320924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48134 (local address 127.5.108.188:33593)
0806 01:06:32.321184 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.321189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.321704 (+ 515us) server_negotiation.cc:407] Connection header received
0806 01:06:32.321976 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.321981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.322077 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.322147 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.323133 (+ 986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48134: BlockingRecv error: recv got EOF from 127.0.0.1:48134 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
I20260806 01:06:32.343798 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.337126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56962 (local address 127.5.108.189:44863)
0806 01:06:32.337329 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.337337 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.337925 (+ 588us) server_negotiation.cc:407] Connection header received
0806 01:06:32.338104 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.338110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.338199 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.338287 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.338771 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.339806 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.341618 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.342296 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.342406 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.342903 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.343066 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.343152 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.343332 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.343469 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:06:32.352458 7209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.350391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48138 (local address 127.5.108.188:33593)
0806 01:06:32.350562 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.350568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.351066 (+ 498us) server_negotiation.cc:407] Connection header received
0806 01:06:32.351218 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.351224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.351307 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.351424 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.352223 (+ 799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48138: BlockingRecv error: recv got EOF from 127.0.0.1:48138 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:06:32.378228 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.372201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56978 (local address 127.5.108.189:44863)
0806 01:06:32.372420 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.372429 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.372753 (+ 324us) server_negotiation.cc:407] Connection header received
0806 01:06:32.372981 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.372985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.373051 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.373129 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.373624 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.374538 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.376538 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.376950 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.377050 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.377567 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.377659 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.377733 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.377908 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.377973 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260806 01:06:32.389299 7209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.383639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48142 (local address 127.5.108.188:33593)
0806 01:06:32.383851 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.383855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.387903 (+ 4048us) server_negotiation.cc:407] Connection header received
0806 01:06:32.388111 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.388115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.388204 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.388326 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.389042 (+ 716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48142: BlockingRecv error: recv got EOF from 127.0.0.1:48142 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:06:32.431819 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.425871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.5.108.189:44863)
0806 01:06:32.426097 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.426108 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.426478 (+ 370us) server_negotiation.cc:407] Connection header received
0806 01:06:32.426634 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.426640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.426744 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.426861 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.427317 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.428523 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.429749 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.430494 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.430626 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.431004 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.431148 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.431249 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.431415 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.431495 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 01:06:32.439818 7209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.437976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48152 (local address 127.5.108.188:33593)
0806 01:06:32.438176 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.438184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.438505 (+ 321us) server_negotiation.cc:407] Connection header received
0806 01:06:32.438637 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.438643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.438764 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.438878 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.439597 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48152: BlockingRecv error: recv got EOF from 127.0.0.1:48152 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:06:32.513195 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.507202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56996 (local address 127.5.108.189:44863)
0806 01:06:32.507471 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.507477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.507842 (+ 365us) server_negotiation.cc:407] Connection header received
0806 01:06:32.508028 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.508032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.508099 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.508178 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.508678 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.509334 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.511242 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.511896 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.512009 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.512476 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.512594 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.512683 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.512853 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.512929 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 01:06:32.520979 7209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:06:32.519412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48166 (local address 127.5.108.188:33593)
0806 01:06:32.519576 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.519579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.519595 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:06:32.519810 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.519815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.519915 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.520020 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:32.520805 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48166: BlockingRecv error: recv got EOF from 127.0.0.1:48166 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:06:32.608515 7229 catalog_manager.cc:1259] Loaded cluster ID: af286bdd26f34ce99515a33b94359aa9
I20260806 01:06:32.608618 7229 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324: loading cluster ID for follower catalog manager: success
I20260806 01:06:32.611012 7229 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324: acquiring CA information for follower catalog manager: success
I20260806 01:06:32.612877 7229 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b887fc3c1807458cbcbc6d7f6f01d324: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:06:32.657218 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.652465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.5.108.189:44863)
0806 01:06:32.652710 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.652716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.652956 (+ 240us) server_negotiation.cc:407] Connection header received
0806 01:06:32.653118 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.653123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.653192 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.653275 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.653695 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.654376 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.655633 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.656030 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.656121 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.656456 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.656589 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.656721 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.656865 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.656956 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:06:32.667534 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.662390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48174 (local address 127.5.108.188:33593)
0806 01:06:32.662629 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.662635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.662902 (+ 267us) server_negotiation.cc:407] Connection header received
0806 01:06:32.663035 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.663039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.663184 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.663306 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.663699 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.664417 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.665604 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.665955 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.666037 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.666430 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.666947 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.667050 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.667141 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.667200 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:06:32.683950 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.672667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.5.108.190:35167)
0806 01:06:32.678055 (+ 5388us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.678062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.678081 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:06:32.678184 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.678190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.678381 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.678569 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.679259 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.680248 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.681686 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.682077 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.682215 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.682837 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.683253 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.683397 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.683559 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.683638 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5265,"thread_start_us":5178,"threads_started":1}
I20260806 01:06:32.684562 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.673150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48178 (local address 127.5.108.188:33593)
0806 01:06:32.673404 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.673412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.678609 (+ 5197us) server_negotiation.cc:407] Connection header received
0806 01:06:32.678903 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.678908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.678999 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.679080 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.679603 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.680289 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.682434 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.683050 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.683167 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.683795 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.683915 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.683990 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.684200 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.684268 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 01:06:32.684829 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:32.672831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.5.108.189:44863)
0806 01:06:32.673051 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:32.673059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:32.677848 (+ 4789us) server_negotiation.cc:407] Connection header received
0806 01:06:32.678127 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:32.678131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:32.678203 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:32.678301 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:32.679086 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.680070 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.682391 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:32.682981 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:32.683127 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:32.683656 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:32.683811 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:32.683874 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:32.684515 (+ 641us) server_negotiation.cc:299] Negotiation successful
0806 01:06:32.684586 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:06:32.689626 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6877
I20260806 01:06:32.690528 6975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:32.798138 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6877
W20260806 01:06:32.826150 7133 connection.cc:535] server connection from 127.0.0.1:48068 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 01:06:32.826620 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 6991
I20260806 01:06:32.827710 7085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:32.928951 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 6991
I20260806 01:06:32.956991 5554 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu pid 7104
I20260806 01:06:32.958042 7198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:33.059389 5554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq1gvHq/build/asan/bin/kudu with pid 7104
2026-08-06T01:06:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4521 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9140 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (370240 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST